brent.edwards
6531440431
CI / build (push) Successful in 24s
CI / lint (push) Successful in 3m16s
CI / quality (push) Successful in 3m45s
CI / typecheck (push) Successful in 3m52s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 4m1s
CI / unit_tests (push) Successful in 7m8s
CI / integration_tests (push) Successful in 7m1s
CI / docker (push) Successful in 1m6s
CI / e2e_tests (push) Successful in 9m49s
CI / coverage (push) Successful in 12m22s
CI / status-check (push) Successful in 1s
CI / benchmark-publish (push) Successful in 19m34s
## Summary Implement the estimation actor as a functional actor type. The estimation actor provides cost, time, and resource estimates for plan operations, running after Strategize completes (before Execute). Estimation is informational only and optional — plans work without an estimation actor configured. ### Changes **New files:** - `src/cleveragents/domain/models/core/estimation.py` — `EstimationResult` frozen Pydantic model with fields for cost (USD), tokens, steps, child plans, time, risk level/factors, summary - `features/estimation_actor.feature` — 12 Behave test scenarios (model validation, serialization, stub actor, plan integration, optional behavior) - `features/steps/estimation_actor_steps.py` — Step definitions - `robot/estimation_actor.robot` — 6 Robot integration tests - `robot/helper_estimation_actor.py` — Robot test helper **Modified files:** - `domain/models/acms/tiers.py` — Added `ESTIMATOR` to `ActorRole` enum - `domain/models/core/plan.py` — Added `estimation_result: EstimationResult | None` field, estimation data in `as_cli_dict()` - `application/services/plan_executor.py` — Added `EstimationStubActor` class - `application/services/plan_lifecycle_service.py` — Added `_run_estimation()` method, invoked in `execute_plan()` - `cli/commands/plan.py` — Display estimation results in plan status - `vulture_whitelist.py` — Added 12 new public symbols ### Quality Gates | Session | Result | |---|---| | `nox -s lint` | PASS | | `nox -s typecheck` | PASS (0 errors) | | `nox -s unit_tests` | PASS (10,818 scenarios) | | `nox -s integration_tests` | PASS (1,512 tests) | | `nox -s coverage_report` | 98% (>= 97%) | Closes #890 Reviewed-on: #962 Co-authored-by: Brent Edwards <brent.edwards@cleverthis.com> Co-committed-by: Brent Edwards <brent.edwards@cleverthis.com>
96 lines
4.2 KiB
Gherkin
96 lines
4.2 KiB
Gherkin
Feature: Estimation Actor
|
|
As a developer
|
|
I want an estimation actor that provides cost, time, and resource estimates
|
|
So that I can make informed decisions before executing plans
|
|
|
|
# EstimationResult domain model
|
|
|
|
Scenario: EstimationResult can be created with all fields populated
|
|
Given I create an estimation result with all fields
|
|
Then the estimation result should have all expected values
|
|
|
|
Scenario: EstimationResult can be created with defaults only
|
|
Given I create an estimation result with defaults
|
|
Then the estimation result summary should be empty
|
|
And the estimation result risk_factors should be empty
|
|
And all optional estimation fields should be None
|
|
|
|
Scenario: EstimationResult as_display_dict includes all populated fields
|
|
Given I create an estimation result with all fields
|
|
When I call as_display_dict on the estimation result
|
|
Then the display dict should include estimated_child_plans
|
|
And the display dict should include estimated_time_seconds
|
|
And the display dict should include risk_factors
|
|
|
|
Scenario: EstimationResult rejects more than 100 risk_factors
|
|
When I try to create an estimation result with 101 risk factors
|
|
Then a validation error should be raised for risk_factors limit
|
|
|
|
Scenario: EstimationResult is frozen (immutable)
|
|
Given I create an estimation result with defaults
|
|
When I try to modify the estimation result summary
|
|
Then a validation error should be raised for frozen model
|
|
|
|
Scenario: EstimationResult can be serialized to dict and back
|
|
Given I create an estimation result with all fields
|
|
When I serialize the estimation result to dict
|
|
And I deserialize it back to an EstimationResult
|
|
Then the deserialized result should match the original
|
|
|
|
# Plan model accepts estimation_result
|
|
|
|
Scenario: Plan model accepts estimation_result field
|
|
Given I have a plan lifecycle service for estimation tests
|
|
And I have an available action for estimation tests
|
|
When I use the action to create a plan for estimation tests
|
|
And I set an estimation result on the plan
|
|
Then the plan estimation_result should be set
|
|
|
|
Scenario: Plan as_cli_dict includes estimation when present
|
|
Given I have a plan lifecycle service for estimation tests
|
|
And I have an available action for estimation tests
|
|
When I use the action to create a plan for estimation tests
|
|
And I set an estimation result on the plan
|
|
Then the plan cli dict should include estimation data
|
|
|
|
# Plans work without estimation actor (optional behavior)
|
|
|
|
Scenario: Plans work without estimation actor configured
|
|
Given I have a plan lifecycle service for estimation tests
|
|
When I create an action without estimation actor for estimation tests
|
|
And I use that action to create an estimation test plan
|
|
Then the plan estimation_actor should be None
|
|
And the plan estimation_result should be None
|
|
|
|
# EstimationStubActor
|
|
|
|
Scenario: EstimationStubActor produces placeholder result
|
|
When I run the estimation stub actor with a valid plan id
|
|
Then the stub result summary should contain "stub actor"
|
|
|
|
Scenario: EstimationStubActor rejects empty plan_id
|
|
When I run the estimation stub actor with an empty plan id
|
|
Then a validation error should be raised for estimation
|
|
|
|
# ActorRole enum includes ESTIMATOR
|
|
|
|
Scenario: ActorRole enum includes ESTIMATOR
|
|
Then the ActorRole enum should have an ESTIMATOR value
|
|
And the ESTIMATOR value should be "estimator"
|
|
|
|
# Estimation invocation in plan lifecycle
|
|
|
|
Scenario: Estimation runs when estimation_actor is set during Strategize to Execute transition
|
|
Given I have a plan lifecycle service for estimation tests
|
|
And I have an action with estimation actor set
|
|
When I use the estimation action and complete strategize
|
|
And I transition the estimation plan to execute phase
|
|
Then the plan estimation_result should be set
|
|
|
|
Scenario: Estimation is skipped when estimation_actor is not set
|
|
Given I have a plan lifecycle service for estimation tests
|
|
And I have an action without estimation actor
|
|
When I use the estimation action and complete strategize
|
|
And I transition the estimation plan to execute phase without estimation
|
|
Then the plan estimation_result should be None
|