Files
cleveragents-core/features/decision_phase_gating.feature
hurui200320 3837327564
CI / lint (push) Successful in 18s
CI / build (push) Successful in 27s
CI / quality (push) Successful in 29s
CI / typecheck (push) Successful in 44s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 53s
CI / unit_tests (push) Successful in 3m14s
CI / integration_tests (push) Successful in 3m38s
CI / docker (push) Successful in 56s
CI / e2e_tests (push) Successful in 5m13s
CI / coverage (push) Successful in 7m0s
CI / benchmark-publish (push) Successful in 19m58s
feat(plan): enforce decision type phase-gating at recording time (#973)
## Summary

Adds phase-gating validation to `DecisionService.record_decision()` that enforces the specification's constraint: certain decision types are only valid during specific plan phases. This prevents invalid decisions (e.g., `tool_invocation` during Strategize, `strategy_choice` during Execute) from being persisted.

### Changes

- **Exception** (`cleveragents.core.exceptions`): Added `DecisionPhaseViolationError(BusinessRuleViolation)` with `decision_type`, `plan_phase`, and `allowed_types` attributes.
- **Phase constants** (`cleveragents.domain.models.core.decision`):
  - `resource_selection` added to `EXECUTE_TYPES` — now phase-agnostic (Strategize or Execute) per ADR-007 L72 and ADR-033 L74.
  - `subplan_spawn` / `subplan_parallel_spawn` in both sets; code comment documents divergence from ADRs per M4 subplan model (ticket #931).
  - `USER_INTERVENTION` remains phase-agnostic (both sets).
  - Module-level docstring table updated to match actual assignments.
  - `is_any_phase_type` property updated to check membership in both sets dynamically (was hardcoded to `USER_INTERVENTION` only).
- **Phase-gating module** (`cleveragents.application.services.phase_gating`):
  - Extracted from `DecisionService` to reduce `decision_service.py` line count (1010 → 913) and isolate the phase-gating concern.
  - `PHASE_ALLOWED_TYPES` typed as `Mapping[PlanPhase, frozenset[DecisionType]]`.
  - `resolve_plan_phase()` helper: supports explicit parameter, DB lookup, and graceful skip.
  - `validate_phase_gating()` enforcement raises `DecisionPhaseViolationError`.
  - Exception narrowing: DB lookup catches `(DatabaseError, OperationalError, OSError)` instead of bare `except Exception` — only absorbs infrastructure failures, not programming errors.
  - `# TODO(pg-migration):` marker on TOCTOU race documentation for future PostgreSQL migration.
- **Decision service** (`cleveragents.application.services.decision_service`):
  - Added `plan_phase` parameter to `record_decision()`.
  - Invalid `plan_phase` string now raises `ValidationError` (was uncaught `ValueError`).
  - Imports and delegates to `phase_gating` module for all phase-gating logic.
  - `PHASE_ALLOWED_TYPES` re-exported in `__all__` for backward compatibility.
- **CHANGELOG**: Added behavioral change entry for `resource_selection` reclassification.
- **Backward compatibility**: Phase-gating is opt-in — when neither `plan_phase` is provided nor a UnitOfWork is wired, validation is skipped, preserving all existing callers.
- **Unrelated drive-by reverted**: Removed `ULID_PATTERN` from `decision.py` `__all__` (was an unrelated export addition).
- **Tests**:
  - 36 Behave scenarios covering valid/invalid types per phase, phase-agnostic acceptance, DB-based resolution (Strategize and Execute plans), unknown plan in DB, PlanPhase enum pass-through, error attributes, and ungated phases.
  - 11 new Behave scenarios for `is_any_phase_type`: 4 dual-phase types (true) + 7 single-phase types (false), including `prompt_definition` root test.
  - 6 Robot Framework integration tests with stderr assertions.
  - Updated `consolidated_decision.feature` for new `EXECUTE_TYPES` member count (8 members).
  - Test cleanup now calls `uow.engine.dispose()` before file deletion.
  - `tempfile.mktemp()` replaced with `tempfile.mkstemp()`.
  - Inline imports moved to module top-level per CONTRIBUTING.md.
  - Flaky concurrency test timing increased in `subplan_execution_steps.py`.

### Review Round 1 + 2 Fixes

| # | Finding | Resolution |
|---|---------|------------|
| P1-1 | `except Exception` too broad in `_resolve_plan_phase` | Narrowed to `(DatabaseError, OperationalError, OSError)` — matches codebase pattern |
| P2-2 | `decision_service.py` at 1010 lines | Extracted to `phase_gating.py` module (1010 → 913 lines) |
| P2-3 | TOCTOU race — no programmatic guard | Added `# TODO(pg-migration):` marker with actionable guidance |
| P2-4 | `resource_selection` reclassification needs CHANGELOG | Added CHANGELOG entry documenting behavioral change |
| P2-7 | `is_any_phase_type` BDD gap for dual-phase types | Added 11 parametrized scenarios covering all 4 dual-phase + 7 single-phase types |
| P3-5 | `ULID_PATTERN` export is unrelated drive-by | Reverted — removed from `decision.py` `__all__` |
| P3-6 | `decision.py` at 514 lines (now 513) | No action — reviewer accepted as marginally over |

### Quality Gates

| Session | Result |
|---------|--------|
| lint | PASS |
| typecheck | PASS (0 errors) |
| unit_tests | PASS (11,153 scenarios, 0 failures) |
| integration_tests | PASS (1,563 tests, 0 failures) |
| e2e_tests | PASS (16 tests, 0 failures) |
| coverage_report | 97% (threshold: 97%) |

Closes #931

Reviewed-on: #973
Co-authored-by: Rui Hu <rui.hu@cleverthis.com>
Co-committed-by: Rui Hu <rui.hu@cleverthis.com>
2026-03-19 07:53:43 +00:00

173 lines
7.2 KiB
Gherkin

Feature: Decision type phase-gating at recording time
As a developer
I want the DecisionService to enforce phase-gating on decision types
So that decisions recorded during Strategize or Execute are constrained
to the types permitted in each phase
Background:
Given a phase-gated decision service
# --- Valid types in Strategize phase ---
Scenario Outline: Valid decision types succeed in Strategize phase
When I record a "<decision_type>" decision in the "strategize" phase
Then the phase-gated decision should be recorded successfully
And the phase-gated decision type should be "<decision_type>"
Examples:
| decision_type |
| prompt_definition |
| invariant_enforced |
| strategy_choice |
| resource_selection |
| subplan_spawn |
| subplan_parallel_spawn |
| user_intervention |
# --- Valid types in Execute phase ---
Scenario Outline: Valid decision types succeed in Execute phase
When I record a "<decision_type>" decision in the "execute" phase
Then the phase-gated decision should be recorded successfully
And the phase-gated decision type should be "<decision_type>"
Examples:
| decision_type |
| implementation_choice |
| resource_selection |
| tool_invocation |
| error_recovery |
| validation_response |
| subplan_spawn |
| subplan_parallel_spawn |
| user_intervention |
# --- Invalid types in Strategize phase ---
Scenario Outline: Execute-only types are rejected in Strategize phase
When I try to record a "<decision_type>" decision in the "strategize" phase
Then a phase violation error should be raised
And the phase violation error decision_type should be "<decision_type>"
And the phase violation error plan_phase should be "strategize"
And the phase violation error allowed_types should not contain "<decision_type>"
Examples:
| decision_type |
| implementation_choice |
| tool_invocation |
| error_recovery |
| validation_response |
# --- Invalid types in Execute phase ---
Scenario Outline: Strategize-only types are rejected in Execute phase
When I try to record a "<decision_type>" decision in the "execute" phase
Then a phase violation error should be raised
And the phase violation error decision_type should be "<decision_type>"
And the phase violation error plan_phase should be "execute"
And the phase violation error allowed_types should not contain "<decision_type>"
Examples:
| decision_type |
| prompt_definition |
| invariant_enforced |
| strategy_choice |
# --- Phase-agnostic types work in both phases ---
Scenario Outline: Phase-agnostic types succeed in both phases
When I record a "<decision_type>" decision in the "strategize" phase
Then the phase-gated decision should be recorded successfully
When I record a "<decision_type>" decision in the "execute" phase
Then the phase-gated decision should be recorded successfully
Examples:
| decision_type |
| resource_selection |
| subplan_spawn |
| subplan_parallel_spawn |
| user_intervention |
# --- Error message quality ---
Scenario: Phase violation error message includes type, phase, and allowed types
When I try to record a "tool_invocation" decision in the "strategize" phase
Then a phase violation error should be raised
And the phase violation error message should contain "tool_invocation"
And the phase violation error message should contain "strategize"
And the phase violation error message should contain "Allowed types"
# --- Phase-gating via database lookup ---
Scenario: Phase resolved from database when plan_phase not explicitly provided
Given a phase-gated decision service with a persisted strategize plan
When I record a "strategy_choice" decision without explicit phase
Then the phase-gated decision should be recorded successfully
When I try to record a "tool_invocation" decision without explicit phase
Then a phase violation error should be raised
Scenario: Phase resolved from database for execute plan
Given a phase-gated decision service with a persisted execute plan
When I record a "tool_invocation" decision without explicit phase
Then the phase-gated decision should be recorded successfully
When I try to record a "strategy_choice" decision without explicit phase
Then a phase violation error should be raised
Scenario: Phase-gating skipped when plan not found in database
Given a phase-gated decision service with an empty database
When I record a "tool_invocation" decision without explicit phase
Then the phase-gated decision should be recorded successfully
# --- No phase-gating when phase is unknown ---
Scenario: Phase-gating is skipped when phase cannot be resolved
Given a phase-gated decision service without persistence
When I record a "tool_invocation" decision without explicit phase
Then the phase-gated decision should be recorded successfully
# --- PHASE_ALLOWED_TYPES constant ---
Scenario: PHASE_ALLOWED_TYPES maps strategize and execute correctly
Then the PHASE_ALLOWED_TYPES should map "strategize" to STRATEGIZE_TYPES
And the PHASE_ALLOWED_TYPES should map "execute" to EXECUTE_TYPES
# --- DecisionPhaseViolationError attributes ---
Scenario: DecisionPhaseViolationError exposes correct attributes
Given a DecisionPhaseViolationError for type "tool_invocation" in phase "strategize"
Then the phase violation error decision_type should be "tool_invocation"
And the phase violation error plan_phase should be "strategize"
And the phase violation error allowed_types should not be empty
# --- _validate_phase_gating static method ---
Scenario: _validate_phase_gating does not raise for ungated phases
When I call _validate_phase_gating with type "tool_invocation" and phase "action"
Then no phase violation error should be raised
When I call _validate_phase_gating with type "tool_invocation" and phase "apply"
Then no phase violation error should be raised
# --- String coercion of plan_phase ---
Scenario: plan_phase accepts a string value
When I record a "strategy_choice" decision with string phase "strategize"
Then the phase-gated decision should be recorded successfully
Scenario: plan_phase accepts a PlanPhase enum directly
When I record a "strategy_choice" decision with PlanPhase enum "strategize"
Then the phase-gated decision should be recorded successfully
# --- Invalid plan_phase string ---
Scenario: Invalid plan_phase string raises ValidationError
When I try to record a "strategy_choice" decision with invalid phase "not_a_real_phase"
Then a validation error should be raised for invalid phase
And the validation error message should contain "not_a_real_phase"
# --- Database error resilience ---
Scenario: resolve_plan_phase gracefully handles database errors
Given a corrupted database unit of work for phase resolution
When I call resolve_plan_phase with the corrupted unit of work
Then resolve_plan_phase should return None