277 lines
11 KiB
Gherkin
277 lines
11 KiB
Gherkin
Feature: Automation Profile Domain Model
|
|
As a developer
|
|
I want automation profile domain models with threshold validation
|
|
So that plan execution autonomy can be configured and enforced
|
|
|
|
# ---- Profile validation: valid thresholds ----
|
|
|
|
Scenario: Profile accepts threshold of 0.0
|
|
When I create a profile with auto_strategize 0.0
|
|
Then the profile model should be created
|
|
And the profile auto_strategize should be 0.0
|
|
|
|
Scenario: Profile accepts threshold of 0.5
|
|
When I create a profile with auto_strategize 0.5
|
|
Then the profile model should be created
|
|
And the profile auto_strategize should be 0.5
|
|
|
|
Scenario: Profile accepts threshold of 1.0
|
|
When I create a profile with auto_strategize 1.0
|
|
Then the profile model should be created
|
|
And the profile auto_strategize should be 1.0
|
|
|
|
# ---- Profile validation: invalid thresholds ----
|
|
|
|
Scenario: Profile rejects threshold below 0.0
|
|
When I try to create a profile with auto_strategize -0.1
|
|
Then a profile validation error should be raised
|
|
And the profile error should mention "greater than or equal"
|
|
|
|
Scenario: Profile rejects threshold above 1.0
|
|
When I try to create a profile with auto_strategize 1.1
|
|
Then a profile validation error should be raised
|
|
And the profile error should mention "less than or equal"
|
|
|
|
Scenario: Profile rejects auto_execute below 0.0
|
|
When I try to create a profile with auto_execute -0.5
|
|
Then a profile validation error should be raised
|
|
|
|
Scenario: Profile rejects auto_execute above 1.0
|
|
When I try to create a profile with auto_execute 2.0
|
|
Then a profile validation error should be raised
|
|
|
|
Scenario: Profile rejects auto_apply below 0.0
|
|
When I try to create a profile with auto_apply -0.01
|
|
Then a profile validation error should be raised
|
|
|
|
Scenario: Profile rejects auto_decisions_strategize above 1.0
|
|
When I try to create a profile with auto_decisions_strategize 1.5
|
|
Then a profile validation error should be raised
|
|
|
|
Scenario: Profile rejects auto_decisions_execute below 0.0
|
|
When I try to create a profile with auto_decisions_execute -1.0
|
|
Then a profile validation error should be raised
|
|
|
|
Scenario: Profile rejects auto_validation_fix above 1.0
|
|
When I try to create a profile with auto_validation_fix 9.9
|
|
Then a profile validation error should be raised
|
|
|
|
Scenario: Profile rejects auto_strategy_revision below 0.0
|
|
When I try to create a profile with auto_strategy_revision -0.001
|
|
Then a profile validation error should be raised
|
|
|
|
Scenario: Profile rejects auto_reversion_from_apply above 1.0
|
|
When I try to create a profile with auto_reversion_from_apply 1.01
|
|
Then a profile validation error should be raised
|
|
|
|
Scenario: Profile rejects auto_child_plans below 0.0
|
|
When I try to create a profile with auto_child_plans -0.5
|
|
Then a profile validation error should be raised
|
|
|
|
Scenario: Profile rejects auto_retry_transient above 1.0
|
|
When I try to create a profile with auto_retry_transient 100.0
|
|
Then a profile validation error should be raised
|
|
|
|
Scenario: Profile rejects auto_checkpoint_restore below 0.0
|
|
When I try to create a profile with auto_checkpoint_restore -0.1
|
|
Then a profile validation error should be raised
|
|
|
|
# ---- Built-in profiles load correctly ----
|
|
|
|
Scenario: Built-in manual profile loads with expected values
|
|
When I load the built-in profile "manual"
|
|
Then the profile model should be created
|
|
And the profile auto_strategize should be 1.0
|
|
And the profile auto_execute should be 1.0
|
|
And the profile auto_apply should be 1.0
|
|
And the profile require_sandbox should be true
|
|
And the profile require_checkpoints should be true
|
|
And the profile allow_unsafe_tools should be false
|
|
|
|
Scenario: Built-in review profile loads with expected values
|
|
When I load the built-in profile "review"
|
|
Then the profile model should be created
|
|
And the profile auto_strategize should be 0.0
|
|
And the profile auto_execute should be 0.0
|
|
And the profile auto_apply should be 1.0
|
|
And the profile auto_child_plans should be 0.0
|
|
|
|
Scenario: Built-in supervised profile loads with expected values
|
|
When I load the built-in profile "supervised"
|
|
Then the profile model should be created
|
|
And the profile auto_strategize should be 0.0
|
|
And the profile auto_execute should be 1.0
|
|
And the profile auto_decisions_strategize should be 0.0
|
|
And the profile auto_child_plans should be 1.0
|
|
|
|
Scenario: Built-in cautious profile loads with expected values
|
|
When I load the built-in profile "cautious"
|
|
Then the profile model should be created
|
|
And the profile auto_strategize should be 0.7
|
|
And the profile auto_execute should be 0.7
|
|
And the profile auto_decisions_strategize should be 0.6
|
|
And the profile auto_decisions_execute should be 0.8
|
|
And the profile auto_validation_fix should be 0.7
|
|
And the profile auto_strategy_revision should be 0.8
|
|
And the profile auto_reversion_from_apply should be 0.9
|
|
And the profile auto_child_plans should be 0.7
|
|
And the profile auto_checkpoint_restore should be 0.6
|
|
|
|
Scenario: Built-in trusted profile loads with expected values
|
|
When I load the built-in profile "trusted"
|
|
Then the profile model should be created
|
|
And the profile auto_strategize should be 0.0
|
|
And the profile auto_execute should be 0.0
|
|
And the profile auto_apply should be 1.0
|
|
And the profile auto_strategy_revision should be 1.0
|
|
And the profile auto_reversion_from_apply should be 1.0
|
|
|
|
Scenario: Built-in auto profile loads with expected values
|
|
When I load the built-in profile "auto"
|
|
Then the profile model should be created
|
|
And the profile auto_strategize should be 0.0
|
|
And the profile auto_apply should be 1.0
|
|
And the profile auto_reversion_from_apply should be 1.0
|
|
And the profile auto_strategy_revision should be 0.0
|
|
|
|
Scenario: Built-in ci profile loads with expected values
|
|
When I load the built-in profile "ci"
|
|
Then the profile model should be created
|
|
And the profile auto_strategize should be 0.0
|
|
And the profile auto_apply should be 0.0
|
|
And the profile require_sandbox should be true
|
|
And the profile allow_unsafe_tools should be false
|
|
|
|
Scenario: Built-in full-auto profile loads with expected values
|
|
When I load the built-in profile "full-auto"
|
|
Then the profile model should be created
|
|
And the profile auto_strategize should be 0.0
|
|
And the profile auto_apply should be 0.0
|
|
And the profile require_sandbox should be false
|
|
And the profile require_checkpoints should be false
|
|
And the profile allow_unsafe_tools should be true
|
|
|
|
# ---- All 8 built-in profiles exist ----
|
|
|
|
Scenario: All 8 built-in profiles are registered
|
|
Then there should be 8 built-in profiles
|
|
And built-in profile "manual" should exist
|
|
And built-in profile "review" should exist
|
|
And built-in profile "supervised" should exist
|
|
And built-in profile "cautious" should exist
|
|
And built-in profile "trusted" should exist
|
|
And built-in profile "auto" should exist
|
|
And built-in profile "ci" should exist
|
|
And built-in profile "full-auto" should exist
|
|
|
|
# ---- Custom profile from YAML dict loads correctly ----
|
|
|
|
Scenario: Custom profile from YAML config loads correctly
|
|
When I load a profile from config with name "acme/strict" and auto_apply 0.8
|
|
Then the profile model should be created
|
|
And the profile name should be "acme/strict"
|
|
And the profile auto_apply should be 0.8
|
|
|
|
Scenario: Profile from config missing name raises error
|
|
When I try to load a profile from config missing name
|
|
Then a profile config error should be raised with "name"
|
|
|
|
# ---- Name format validation ----
|
|
|
|
Scenario: Profile name accepts bare name
|
|
When I create a profile with name "manual"
|
|
Then the profile model should be created
|
|
And the profile name should be "manual"
|
|
|
|
Scenario: Profile name accepts namespaced name
|
|
When I create a profile with name "acme/strict"
|
|
Then the profile model should be created
|
|
And the profile name should be "acme/strict"
|
|
|
|
Scenario: Profile name rejects spaces
|
|
When I try to create a profile with invalid name "bad name"
|
|
Then a profile validation error should be raised
|
|
And the profile error should mention "Profile name"
|
|
|
|
Scenario: Profile name rejects empty string
|
|
When I try to create a profile with empty name
|
|
Then a profile validation error should be raised
|
|
|
|
Scenario: Profile name rejects double slash
|
|
When I try to create a profile with invalid name "bad//name"
|
|
Then a profile validation error should be raised
|
|
|
|
Scenario: Profile name rejects trailing slash
|
|
When I try to create a profile with invalid name "bad/"
|
|
Then a profile validation error should be raised
|
|
|
|
# ---- get_builtin_profile helper ----
|
|
|
|
Scenario: get_builtin_profile returns correct profile
|
|
When I call get_builtin_profile with "ci"
|
|
Then the profile model should be created
|
|
And the profile name should be "ci"
|
|
|
|
Scenario: get_builtin_profile raises for unknown name
|
|
When I try to call get_builtin_profile with "nonexistent"
|
|
Then a profile key error should be raised
|
|
|
|
# ---- Schema version ----
|
|
|
|
Scenario: Profile has default schema version
|
|
When I create a profile with name "test-profile"
|
|
Then the profile schema_version should be "1.0"
|
|
|
|
Scenario: Profile accepts custom schema version
|
|
When I create a profile with schema_version "2.0"
|
|
Then the profile schema_version should be "2.0"
|
|
|
|
# ---- Safety field defaults ----
|
|
|
|
Scenario: Profile safety fields have correct defaults
|
|
When I create a profile with name "test-defaults"
|
|
Then the profile require_sandbox should be true
|
|
And the profile require_checkpoints should be true
|
|
And the profile allow_unsafe_tools should be false
|
|
|
|
# ---- Description field ----
|
|
|
|
Scenario: Profile description defaults to empty string
|
|
When I create a profile with name "no-desc"
|
|
Then the profile description should be empty
|
|
|
|
Scenario: Profile description accepts custom value
|
|
When I create a profile with description "My custom profile"
|
|
Then the profile description should be "My custom profile"
|
|
|
|
# ---- validate_assignment enforcement ----
|
|
|
|
Scenario: Assigning invalid threshold raises error
|
|
When I create a profile with name "assign-test"
|
|
And I try to assign auto_strategize 1.5 on the profile
|
|
Then a profile validation error should be raised
|
|
|
|
# ---- Built-in profile retry and checkpoint values ----
|
|
|
|
Scenario: Manual profile has auto_retry_transient 1.0
|
|
When I load the built-in profile "manual"
|
|
Then the profile auto_retry_transient should be 1.0
|
|
And the profile auto_checkpoint_restore should be 1.0
|
|
|
|
Scenario: Review profile has auto_retry_transient 0.0
|
|
When I load the built-in profile "review"
|
|
Then the profile auto_retry_transient should be 0.0
|
|
And the profile auto_checkpoint_restore should be 1.0
|
|
|
|
Scenario: Cautious profile has auto_retry_transient 0.0
|
|
When I load the built-in profile "cautious"
|
|
Then the profile auto_retry_transient should be 0.0
|
|
|
|
# ---- Model dump and round-trip ----
|
|
|
|
Scenario: Profile model_dump produces valid dict
|
|
When I create a profile and dump it with auto_apply 0.5
|
|
Then the profile model dump should have key "name"
|
|
And the profile model dump should have key "auto_apply"
|
|
And the profile model dump auto_apply should be 0.5
|