HAL9000
8ed8c25652
The Strategize phase was recording decisions with minimal context snapshots (only a hash of question+chosen_option), violating the v3.2.0 acceptance criterion that decisions must include full context snapshots sufficient to replay the decision. Changes: - Add _build_strategize_context_snapshot() helper that builds a full ContextSnapshot from plan metadata (description, action_name, strategy_actor, project_links) - Update _try_record_decision() to accept an optional context_snapshot parameter and forward it to DecisionService - Update start_strategize() to build and pass a full context snapshot - Add 3 BDD scenarios in decision_recording.feature verifying that hot_context_hash, hot_context_ref, actor_state_ref, and relevant_resources are all populated for Strategize-phase decisions ISSUES CLOSED: #9056
433 lines
22 KiB
Gherkin
433 lines
22 KiB
Gherkin
Feature: Decision recording and snapshot store
|
|
As a developer
|
|
I want a DecisionService that records decisions with auto-sequencing and snapshots
|
|
So that plan decision trees are tracked with full context for replay
|
|
|
|
Background:
|
|
Given a decision service
|
|
|
|
# --- Recording ---
|
|
|
|
Scenario: Record a root decision with auto-sequencing
|
|
When I record a prompt_definition decision for plan "P1" with question "What is the plan prompt?"
|
|
Then the dsvc decision should be recorded successfully
|
|
And the dsvc decision sequence number should be 0
|
|
And the dsvc decision type should be "prompt_definition"
|
|
And the dsvc decision should be a root decision
|
|
And the dsvc decision should have a valid ULID as decision_id
|
|
And the dsvc decision context snapshot hash should not be empty
|
|
|
|
Scenario: Record multiple decisions with monotonic sequencing
|
|
When I record a prompt_definition decision for plan "P1" with question "Plan prompt"
|
|
And I record a strategy_choice decision for plan "P1" with question "Which approach?"
|
|
And I record an implementation_choice decision for plan "P1" with question "How to implement?"
|
|
Then the dsvc plan "P1" should have 3 decisions
|
|
And the dsvc decisions should have sequence numbers 0 1 2
|
|
|
|
Scenario: Record a decision with explicit context snapshot
|
|
Given a context snapshot with hash "sha256:abc123" and ref "s3://bucket/ctx"
|
|
When I record a strategy_choice decision for plan "P1" with the explicit snapshot
|
|
Then the dsvc decision context snapshot hash should be "sha256:abc123"
|
|
And the dsvc decision context snapshot ref should be "s3://bucket/ctx"
|
|
|
|
Scenario: Record a decision with confidence score
|
|
When I record a strategy_choice decision for plan "P1" with confidence 0.85
|
|
Then the dsvc decision confidence score should be 0.85
|
|
|
|
Scenario: Record a correction decision
|
|
When I record a prompt_definition decision for plan "P1" with question "Original"
|
|
And I record a correction decision for plan "P1" correcting the first decision
|
|
Then the dsvc correction decision is_correction should be true
|
|
And the dsvc correction decision corrects_decision_id should match the first decision
|
|
|
|
Scenario: Record a decision with alternatives
|
|
When I record a strategy_choice decision for plan "P1" with alternatives "Option A" "Option B" "Option C"
|
|
Then the dsvc decision should have 3 alternatives considered
|
|
|
|
Scenario: Record a decision with artifacts
|
|
When I record an implementation_choice decision for plan "P1" with artifacts
|
|
Then the dsvc decision should have 2 artifacts produced
|
|
|
|
Scenario: Recording with empty plan_id raises ValidationError
|
|
When I try to record a decision with empty plan_id
|
|
Then a dsvc validation error should be raised
|
|
And the dsvc error should mention "plan_id"
|
|
|
|
Scenario: Recording with empty question raises ValidationError
|
|
When I try to record a decision with empty question
|
|
Then a dsvc validation error should be raised
|
|
And the dsvc error should mention "question"
|
|
|
|
Scenario: Recording with empty chosen_option raises ValidationError
|
|
When I try to record a decision with empty chosen_option
|
|
Then a dsvc validation error should be raised
|
|
And the dsvc error should mention "chosen_option"
|
|
|
|
Scenario: Recording with invalid decision_type raises ValueError
|
|
When I try to record a decision with invalid type "nonexistent_type"
|
|
Then a dsvc value error should be raised
|
|
And the dsvc error should mention "nonexistent_type"
|
|
|
|
Scenario: Recording with confidence_score at boundaries
|
|
When I record a strategy_choice decision for plan "P1" with confidence 0.0
|
|
Then the dsvc decision confidence score should be 0.0
|
|
When I record a strategy_choice decision for plan "P1" with confidence 1.0
|
|
Then the dsvc decision confidence score should be 1.0
|
|
|
|
Scenario: Recording with negative confidence_score raises ValidationError
|
|
When I try to record a decision with confidence -0.1
|
|
Then a dsvc validation error should be raised
|
|
|
|
Scenario: Recording with confidence_score above 1.0 raises ValidationError
|
|
When I try to record a decision with confidence 1.1
|
|
Then a dsvc validation error should be raised
|
|
|
|
Scenario: Auto-capture generates a snapshot hash
|
|
When I record a strategy_choice decision for plan "P1" with question "Test auto-capture"
|
|
Then the dsvc decision context snapshot hash should start with "sha256:"
|
|
|
|
# --- Retrieval ---
|
|
|
|
Scenario: Get a decision by ID
|
|
When I record a prompt_definition decision for plan "P1" with question "Test get"
|
|
And I get the decision by its ID
|
|
Then the dsvc retrieved decision should match the recorded decision
|
|
|
|
Scenario: Get a non-existent decision raises DecisionNotFoundError
|
|
When I try to get decision "01NONEXISTENT00000000000000"
|
|
Then a dsvc decision not found error should be raised
|
|
|
|
Scenario: List decisions for a plan returns all decisions ordered
|
|
When I record a prompt_definition decision for plan "P1" with question "First"
|
|
And I record a strategy_choice decision for plan "P1" with question "Second"
|
|
And I record an implementation_choice decision for plan "P1" with question "Third"
|
|
And I list decisions for plan "P1"
|
|
Then the dsvc decision list should have 3 entries
|
|
And the dsvc decision list should be ordered by sequence number
|
|
|
|
Scenario: List decisions for empty plan returns empty list
|
|
When I list decisions for plan "P_EMPTY"
|
|
Then the dsvc decision list should have 0 entries
|
|
|
|
Scenario: List decisions by type filters correctly
|
|
When I record a prompt_definition decision for plan "P1" with question "Root"
|
|
And I record a strategy_choice decision for plan "P1" with question "Strategy 1"
|
|
And I record a strategy_choice decision for plan "P1" with question "Strategy 2"
|
|
And I record an implementation_choice decision for plan "P1" with question "Impl"
|
|
And I filter decisions for plan "P1" by type "strategy_choice"
|
|
Then the dsvc decision list should have 2 entries
|
|
|
|
# --- Tree operations ---
|
|
|
|
Scenario: Get tree returns BFS order from root
|
|
When I record a prompt_definition decision for plan "P1" with question "Root"
|
|
And I record a strategy_choice decision for plan "P1" with parent as child "Child 1"
|
|
And I record a strategy_choice decision for plan "P1" with parent as child "Child 2"
|
|
And I record an implementation_choice decision for plan "P1" with second parent as child "Grandchild"
|
|
And I get the tree for plan "P1"
|
|
Then the dsvc tree should have 4 decisions
|
|
And the dsvc first tree decision should be a root decision
|
|
And the dsvc tree should be in BFS level order
|
|
|
|
Scenario: Get tree for empty plan returns empty list
|
|
When I get the tree for plan "P_EMPTY"
|
|
Then the dsvc tree should have 0 decisions
|
|
|
|
Scenario: Get path to root navigates upward
|
|
When I record a prompt_definition decision for plan "P1" with question "Root"
|
|
And I record a strategy_choice decision for plan "P1" with parent as child "Level 1"
|
|
And I record an implementation_choice decision for plan "P1" with second parent as child "Level 2"
|
|
And I get the path to root from the last decision
|
|
Then the dsvc path should have 3 decisions
|
|
And the dsvc first path decision should be the last recorded decision
|
|
And the dsvc last path decision should be the root
|
|
|
|
Scenario: Get path to root for non-existent decision raises error
|
|
When I try to get path to root for decision "01NONEXISTENT00000000000000"
|
|
Then a dsvc decision not found error should be raised
|
|
|
|
# --- Superseded decisions ---
|
|
|
|
Scenario: Mark a decision as superseded
|
|
When I record a prompt_definition decision for plan "P1" with question "Original"
|
|
And I record a strategy_choice decision for plan "P1" with question "Replacement"
|
|
And I mark the first decision as superseded by the second
|
|
Then the dsvc first decision should be superseded
|
|
And the dsvc first decision superseded_by should match the second decision
|
|
|
|
Scenario: Get superseded decisions for a plan
|
|
When I record a prompt_definition decision for plan "P1" with question "Original"
|
|
And I record a strategy_choice decision for plan "P1" with question "Replacement"
|
|
And I mark the first decision as superseded by the second
|
|
And I get superseded decisions for plan "P1"
|
|
Then the dsvc superseded list should have 1 entry
|
|
|
|
Scenario: Mark non-existent decision raises DecisionNotFoundError
|
|
When I try to mark decision "01NONEXISTENT00000000000000" as superseded
|
|
Then a dsvc decision not found error should be raised
|
|
|
|
# --- Delete ---
|
|
|
|
Scenario: Delete a decision
|
|
When I record a prompt_definition decision for plan "P1" with question "To delete"
|
|
And I delete the recorded decision
|
|
Then the dsvc decision should be deleted
|
|
And the dsvc plan "P1" should have 0 decisions
|
|
|
|
Scenario: Delete non-existent decision raises DecisionNotFoundError
|
|
When I try to delete decision "01NONEXISTENT00000000000000"
|
|
Then a dsvc decision not found error should be raised
|
|
|
|
# --- Snapshot store ---
|
|
|
|
Scenario: Snapshot is stored when decision is recorded
|
|
When I record a strategy_choice decision for plan "P1" with question "Snapshot test"
|
|
And I get the snapshot for the recorded decision
|
|
Then the dsvc snapshot should not be None
|
|
|
|
Scenario: Get snapshots for plan returns all snapshots
|
|
When I record a prompt_definition decision for plan "P1" with question "First"
|
|
And I record a strategy_choice decision for plan "P1" with question "Second"
|
|
And I get snapshots for plan "P1"
|
|
Then the dsvc snapshots dict should have 2 entries
|
|
|
|
Scenario: Snapshot hash deduplication index works
|
|
Given a context snapshot with hash "sha256:samehash" and ref "ref1"
|
|
When I record a strategy_choice decision for plan "P1" with the explicit snapshot
|
|
And I record a strategy_choice decision for plan "P2" with the same explicit snapshot
|
|
And I query the snapshot store by hash "sha256:samehash"
|
|
Then the dsvc hash query should return 2 decision IDs
|
|
|
|
Scenario: Snapshot is removed when decision is deleted
|
|
When I record a prompt_definition decision for plan "P1" with question "Ephemeral"
|
|
And I delete the recorded decision
|
|
And I get the snapshot for the deleted decision
|
|
Then the dsvc snapshot should be None
|
|
|
|
# --- Statistics ---
|
|
|
|
Scenario: Count decisions for a plan
|
|
When I record a prompt_definition decision for plan "P1" with question "One"
|
|
And I record a strategy_choice decision for plan "P1" with question "Two"
|
|
Then the dsvc decision count for plan "P1" should be 2
|
|
|
|
Scenario: Get next sequence returns the expected value
|
|
Then the dsvc next sequence for plan "P_NEW" should be 0
|
|
When I record a prompt_definition decision for plan "P_NEW" with question "First"
|
|
Then the dsvc next sequence for plan "P_NEW" should be 1
|
|
|
|
# --- DuplicateDecisionError ---
|
|
|
|
Scenario: DuplicateDecisionError contains decision_id
|
|
Given a DuplicateDecisionError for decision "DUP123"
|
|
Then the dsvc duplicate error decision_id should be "DUP123"
|
|
And the dsvc duplicate error message should contain "DUP123"
|
|
|
|
# --- SequenceConflictError ---
|
|
|
|
Scenario: SequenceConflictError contains plan_id and sequence
|
|
Given a SequenceConflictError for plan "P1" sequence 5
|
|
Then the dsvc sequence error plan_id should be "P1"
|
|
And the dsvc sequence error sequence_number should be 5
|
|
And the dsvc sequence error message should contain "P1"
|
|
|
|
# --- SnapshotStore standalone ---
|
|
|
|
Scenario: SnapshotStore remove returns False for unknown decision
|
|
Given a standalone snapshot store
|
|
When I remove snapshot for decision "UNKNOWN"
|
|
Then the dsvc snapshot remove result should be False
|
|
|
|
Scenario: SnapshotStore get returns None for unknown decision
|
|
Given a standalone snapshot store
|
|
When I get snapshot for decision "UNKNOWN"
|
|
Then the dsvc standalone snapshot should be None
|
|
|
|
Scenario: SnapshotStore get_by_hash returns empty for unknown hash
|
|
Given a standalone snapshot store
|
|
When I query the standalone store by hash "sha256:nope"
|
|
Then the dsvc standalone hash query should return 0 decision IDs
|
|
|
|
Scenario: DecisionService string type coercion works
|
|
When I record a decision for plan "P1" with string type "strategy_choice"
|
|
Then the dsvc decision type should be "strategy_choice"
|
|
|
|
# --- Edge-case coverage ---
|
|
|
|
Scenario: Snapshot stored without a hash skips hash index
|
|
Given a standalone snapshot store
|
|
And a context snapshot with no hash
|
|
When I store the hashless snapshot for decision "D1"
|
|
And I get standalone snapshot for decision "D1"
|
|
Then the dsvc standalone retrieved snapshot should not be None
|
|
And the dsvc standalone retrieved snapshot hash should be empty
|
|
|
|
Scenario: Snapshot store remove cleans up hash index
|
|
Given a standalone snapshot store
|
|
And a context snapshot with hash "sha256:removeme" and ref "ref"
|
|
When I store the explicit snapshot for decision "D_REM"
|
|
And I remove snapshot for decision "D_REM"
|
|
Then the dsvc snapshot remove result should be True
|
|
And the dsvc standalone hash query for "sha256:removeme" should return 0 decision IDs
|
|
|
|
Scenario: Snapshot store remove last entry deletes hash bucket
|
|
Given a standalone snapshot store
|
|
And a context snapshot with hash "sha256:onlyone" and ref "ref"
|
|
When I store the explicit snapshot for decision "D_ONLY"
|
|
And I remove snapshot for decision "D_ONLY"
|
|
Then the dsvc snapshot remove result should be True
|
|
And the dsvc standalone hash query for "sha256:onlyone" should return 0 decision IDs
|
|
|
|
Scenario: Snapshot store remove snapshot with no hash
|
|
Given a standalone snapshot store
|
|
And a context snapshot with no hash
|
|
When I store the hashless snapshot for decision "D_NOHASH"
|
|
And I remove snapshot for decision "D_NOHASH"
|
|
Then the dsvc snapshot remove result should be True
|
|
|
|
Scenario: Snapshot store remove with multiple entries keeps remaining
|
|
Given a standalone snapshot store
|
|
And a context snapshot with hash "sha256:shared" and ref "ref"
|
|
When I store the explicit snapshot for decision "D_KEEP1"
|
|
And I store the explicit snapshot for decision "D_KEEP2"
|
|
And I remove snapshot for decision "D_KEEP1"
|
|
Then the dsvc snapshot remove result should be True
|
|
And the dsvc standalone hash query for "sha256:shared" should return 1 decision IDs
|
|
|
|
Scenario: Get tree when no decision is a root returns all decisions
|
|
When I record a decision for plan "P1" with a forced non-root parent
|
|
And I get the tree for plan "P1"
|
|
Then the dsvc tree should have 1 decisions
|
|
|
|
Scenario: List by type with string coercion filters correctly
|
|
When I record a prompt_definition decision for plan "P1" with question "Type1"
|
|
And I record a strategy_choice decision for plan "P1" with question "Type2"
|
|
And I filter decisions for plan "P1" by type "prompt_definition"
|
|
Then the dsvc decision list should have 1 entries
|
|
|
|
Scenario: Recording with whitespace-only plan_id raises ValidationError
|
|
When I try to record a decision with whitespace plan_id
|
|
Then a dsvc validation error should be raised
|
|
And the dsvc error should mention "plan_id"
|
|
|
|
Scenario: Recording with whitespace-only question raises ValidationError
|
|
When I try to record a decision with whitespace question
|
|
Then a dsvc validation error should be raised
|
|
And the dsvc error should mention "question"
|
|
|
|
Scenario: Recording with whitespace-only chosen_option raises ValidationError
|
|
When I try to record a decision with whitespace chosen_option
|
|
Then a dsvc validation error should be raised
|
|
And the dsvc error should mention "chosen_option"
|
|
|
|
Scenario: SnapshotStore list_for_plan skips missing snapshots
|
|
Given a standalone snapshot store
|
|
When I list snapshots for decisions with missing entries
|
|
Then the dsvc standalone plan snapshots should have 0 entries
|
|
|
|
# --- Persisted-mode (database-backed) ---
|
|
|
|
Scenario: Persisted record and get decision round-trips through database
|
|
Given a persisted decision service
|
|
When I record a prompt_definition decision for plan "P1" with question "Persisted root"
|
|
And I get the decision by its ID
|
|
Then the dsvc retrieved decision should match the recorded decision
|
|
|
|
Scenario: Persisted get non-existent decision raises DecisionNotFoundError
|
|
Given a persisted decision service
|
|
When I try to get decision "01NONEXISTENT00000000000000"
|
|
Then a dsvc decision not found error should be raised
|
|
|
|
Scenario: Persisted list decisions returns ordered results from database
|
|
Given a persisted decision service
|
|
When I record a prompt_definition decision for plan "P1" with question "First persisted"
|
|
And I record a strategy_choice decision for plan "P1" with question "Second persisted"
|
|
And I list decisions for plan "P1"
|
|
Then the dsvc decision list should have 2 entries
|
|
And the dsvc decision list should be ordered by sequence number
|
|
|
|
Scenario: Persisted list by type filters through database
|
|
Given a persisted decision service
|
|
When I record a prompt_definition decision for plan "P1" with question "Root"
|
|
And I record a strategy_choice decision for plan "P1" with question "Strategy"
|
|
And I record a strategy_choice decision for plan "P1" with question "Strategy 2"
|
|
And I filter decisions for plan "P1" by type "strategy_choice"
|
|
Then the dsvc decision list should have 2 entries
|
|
|
|
Scenario: Persisted get path to root navigates through database
|
|
Given a persisted decision service
|
|
When I record a prompt_definition decision for plan "P1" with question "Root"
|
|
And I record a strategy_choice decision for plan "P1" with parent as child "Level 1"
|
|
And I record an implementation_choice decision for plan "P1" with second parent as child "Level 2"
|
|
And I get the path to root from the last decision
|
|
Then the dsvc path should have 3 decisions
|
|
And the dsvc last path decision should be the root
|
|
|
|
Scenario: Persisted get superseded returns superseded decisions from database
|
|
Given a persisted decision service
|
|
When I record a prompt_definition decision for plan "P1" with question "Original"
|
|
And I record a strategy_choice decision for plan "P1" with question "Replacement"
|
|
And I mark the first decision as superseded by the second
|
|
And I get superseded decisions for plan "P1"
|
|
Then the dsvc superseded list should have 1 entry
|
|
|
|
Scenario: Persisted mark superseded updates database and cache
|
|
Given a persisted decision service
|
|
When I record a prompt_definition decision for plan "P1" with question "Old"
|
|
And I record a strategy_choice decision for plan "P1" with question "New"
|
|
And I mark the first decision as superseded by the second
|
|
Then the dsvc first decision should be superseded
|
|
And the dsvc first decision superseded_by should match the second decision
|
|
|
|
Scenario: Persisted delete removes decision from database
|
|
Given a persisted decision service
|
|
When I record a prompt_definition decision for plan "P1" with question "Ephemeral"
|
|
And I delete the recorded decision
|
|
Then the dsvc decision should be deleted
|
|
And the dsvc plan "P1" should have 0 decisions
|
|
|
|
Scenario: Persisted duplicate decision raises DuplicateDecisionError
|
|
Given a persisted decision service
|
|
When I record a prompt_definition decision for plan "P1" with question "Original"
|
|
And I try to store a duplicate of the recorded decision
|
|
Then a dsvc duplicate decision error should be raised
|
|
|
|
Scenario: Persisted sequence resumes after service restart
|
|
Given a persisted decision service
|
|
When I record a prompt_definition decision for plan "P1" with question "Before restart"
|
|
And I record a strategy_choice decision for plan "P1" with question "Also before restart"
|
|
Then the dsvc decisions should have sequence numbers 0 1
|
|
When I recreate the decision service with the same database
|
|
And I record a strategy_choice decision for plan "P1" with question "After restart"
|
|
Then the dsvc decision sequence number should be 2
|
|
And the dsvc next sequence for plan "P1" should be 3
|
|
|
|
# --- Full context snapshot (issue #9056) ---
|
|
|
|
Scenario: Strategize phase records decisions with full context snapshots
|
|
Given a plan lifecycle service with decision service wired
|
|
And an action "local/test-action" for strategize snapshot test
|
|
And a plan created from "local/test-action" with project "proj-snapshot"
|
|
When I start strategize for the snapshot test plan
|
|
Then the strategize decision should have a non-empty hot_context_hash
|
|
And the strategize decision should have a non-empty hot_context_ref
|
|
And the strategize decision hot_context_ref should start with "plan:"
|
|
And the strategize decision should have a non-empty actor_state_ref
|
|
And the strategize decision should have relevant_resources populated
|
|
|
|
Scenario: Strategize context snapshot hash is content-addressable
|
|
Given a plan lifecycle service with decision service wired
|
|
And an action "local/test-action-hash" for strategize snapshot test
|
|
And a plan created from "local/test-action-hash" with project "proj-hash"
|
|
When I start strategize for the snapshot test plan
|
|
Then the strategize decision hot_context_hash should start with "sha256:"
|
|
And the strategize decision hot_context_hash should be 71 characters long
|
|
|
|
Scenario: Strategize context snapshot without projects has empty relevant_resources
|
|
Given a plan lifecycle service with decision service wired
|
|
And an action "local/no-project-action" for strategize snapshot test
|
|
And a plan created from "local/no-project-action" without projects
|
|
When I start strategize for the snapshot test plan
|
|
Then the strategize decision should have a non-empty hot_context_hash
|
|
And the strategize decision should have empty relevant_resources
|