cleveragents/cleveragents-core

Fork 3

UAT: Plan estimation uses `EstimationStubActor` — real actor registry dispatch is never invoked for cost estimation #4144

New issue

Open

opened 2026-04-06 10:50:41 +00:00 by freemo · 2 comments

freemo commented

2026-04-06 10:50:41 +00:00

Owner

Metadata

Branch: fix/plan-estimation-real-actor-dispatch
Commit Message: fix(plan): replace EstimationStubActor with real actor registry dispatch
Milestone: (none — backlog)
Parent Epic: #3374 (E2E Workflow Specification Tests & Code Review Tool Examples Epic)

Bug Report

What Was Tested

Plan cost estimation — the ability to estimate the cost of a plan before execution using a configured estimation actor.

Expected Behavior (from spec)

Per docs/specification.md §Plan Lifecycle:

Each profile composes a Safety Profile that controls hard safety constraints (sandbox, checkpoint, unsafe-tool gating, skill restrictions, cost/retry limits).

The spec describes an estimation_actor field on Actions and Plans that should be invoked to produce cost estimates. When configured, the estimation actor should be dispatched via the actor registry to produce a real EstimationResult.

Actual Behavior

_run_estimation() in src/cleveragents/application/services/plan_lifecycle_service.py (line 343) uses a hardcoded stub instead of dispatching to the actor registry:

def _run_estimation(self, plan: Plan) -> None:
    """Run the estimation actor if configured on the plan."""
    actor_name = plan.estimation_actor
    if not actor_name or actor_name == "__optional__":
        return

    try:
        from cleveragents.application.services.plan_executor import (
            EstimationStubActor,
        )

        # TODO: Replace EstimationStubActor with real actor dispatch
        # via actor registry
        stub = EstimationStubActor()
        result = stub.estimate(plan.identity.plan_id)

The EstimationStubActor returns a fixed stub result regardless of the plan's actual content or the configured actor.

Impact

Cost estimation is non-functional — all estimates are stub values
The --estimation-actor flag on agents plan use has no effect on estimation quality
Users cannot get meaningful cost estimates before executing plans
The cost_estimate_usd field on plans is always a stub value
Safety profiles with max_cost_per_plan limits cannot be properly enforced without real estimates

Code Location

src/cleveragents/application/services/plan_lifecycle_service.py — _run_estimation() method, line 343
TODO comment: "Replace EstimationStubActor with real actor dispatch via actor registry"
src/cleveragents/application/services/plan_executor.py — EstimationStubActor class

Steps to Reproduce

Create an action with estimation_actor: local/my-estimator
Use the action: agents plan use local/my-action local/my-project --estimation-actor local/my-estimator
Check the plan's cost estimate
Observe that the estimate is a stub value, not from the configured actor

Subtasks

Replace EstimationStubActor usage with real actor registry dispatch in _run_estimation()
Use LLMActorsService or equivalent to invoke the configured estimation actor
Pass the plan's description and context to the estimation actor
Parse the actor's response into an EstimationResult
Add BDD scenario verifying real estimation actor dispatch (with mock actor)
Remove EstimationStubActor from plan_executor.py once real dispatch is wired

Definition of Done

_run_estimation() dispatches to the configured actor via the actor registry
The estimation actor receives the plan's description and context
The EstimationResult is populated from the actor's response
EstimationStubActor is removed or deprecated
All nox stages pass
Coverage >= 97%

Backlog note: This issue was discovered during autonomous operation
on milestone v3.6.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/plan-estimation-real-actor-dispatch` - **Commit Message**: `fix(plan): replace EstimationStubActor with real actor registry dispatch` - **Milestone**: _(none — backlog)_ - **Parent Epic**: #3374 (E2E Workflow Specification Tests & Code Review Tool Examples Epic) ## Bug Report ### What Was Tested Plan cost estimation — the ability to estimate the cost of a plan before execution using a configured estimation actor. ### Expected Behavior (from spec) Per `docs/specification.md` §Plan Lifecycle: > Each profile composes a **Safety Profile** that controls hard safety constraints (sandbox, checkpoint, unsafe-tool gating, skill restrictions, cost/retry limits). The spec describes an `estimation_actor` field on Actions and Plans that should be invoked to produce cost estimates. When configured, the estimation actor should be dispatched via the actor registry to produce a real `EstimationResult`. ### Actual Behavior `_run_estimation()` in `src/cleveragents/application/services/plan_lifecycle_service.py` (line 343) uses a hardcoded stub instead of dispatching to the actor registry: ```python def _run_estimation(self, plan: Plan) -> None: """Run the estimation actor if configured on the plan.""" actor_name = plan.estimation_actor if not actor_name or actor_name == "__optional__": return try: from cleveragents.application.services.plan_executor import ( EstimationStubActor, ) # TODO: Replace EstimationStubActor with real actor dispatch # via actor registry stub = EstimationStubActor() result = stub.estimate(plan.identity.plan_id) ``` The `EstimationStubActor` returns a fixed stub result regardless of the plan's actual content or the configured actor. ### Impact - Cost estimation is non-functional — all estimates are stub values - The `--estimation-actor` flag on `agents plan use` has no effect on estimation quality - Users cannot get meaningful cost estimates before executing plans - The `cost_estimate_usd` field on plans is always a stub value - Safety profiles with `max_cost_per_plan` limits cannot be properly enforced without real estimates ### Code Location - `src/cleveragents/application/services/plan_lifecycle_service.py` — `_run_estimation()` method, line 343 - TODO comment: "Replace EstimationStubActor with real actor dispatch via actor registry" - `src/cleveragents/application/services/plan_executor.py` — `EstimationStubActor` class ### Steps to Reproduce 1. Create an action with `estimation_actor: local/my-estimator` 2. Use the action: `agents plan use local/my-action local/my-project --estimation-actor local/my-estimator` 3. Check the plan's cost estimate 4. Observe that the estimate is a stub value, not from the configured actor ## Subtasks - [ ] Replace `EstimationStubActor` usage with real actor registry dispatch in `_run_estimation()` - [ ] Use `LLMActorsService` or equivalent to invoke the configured estimation actor - [ ] Pass the plan's description and context to the estimation actor - [ ] Parse the actor's response into an `EstimationResult` - [ ] Add BDD scenario verifying real estimation actor dispatch (with mock actor) - [ ] Remove `EstimationStubActor` from `plan_executor.py` once real dispatch is wired ## Definition of Done - [ ] `_run_estimation()` dispatches to the configured actor via the actor registry - [ ] The estimation actor receives the plan's description and context - [ ] The `EstimationResult` is populated from the actor's response - [ ] `EstimationStubActor` is removed or deprecated - [ ] All nox stages pass - [ ] Coverage >= 97% > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.6.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator

freemo added a new dependency

2026-04-06 10:50:54 +00:00

#3374 Epic: E2E Workflow Specification Tests & Code Review Tool Examples

freemo referenced this issue

2026-04-06 10:51:15 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo referenced this issue

2026-04-06 10:51:36 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo added the

Points

label

2026-04-06 10:53:49 +00:00

freemo commented

2026-04-06 16:18:01 +00:00

Author

Owner

TRIAGE: v3.1.0 Models & Schema - High Priority

This automated issue identifies critical missing functionality: real cost estimation is not working, only stub values are returned. This impacts cost controls and safety profiles.

Rationale:

Cost estimation is core to safety profiles and budget controls
Currently all cost estimates are stub values, making cost controls ineffective
Labeled Priority/Backlog but should be elevated for working cost management
Essential for any production use where cost controls matter
Affects safety profile enforcement (max_cost_per_plan limits)

Recommendation: Upgrade from Priority/Backlog to Priority/High and assign to v3.1.0 as this represents missing core functionality needed for a minimal working cost management system.

**TRIAGE: v3.1.0 Models & Schema - High Priority** This automated issue identifies critical missing functionality: real cost estimation is not working, only stub values are returned. This impacts cost controls and safety profiles. **Rationale:** - Cost estimation is core to safety profiles and budget controls - Currently all cost estimates are stub values, making cost controls ineffective - Labeled Priority/Backlog but should be elevated for working cost management - Essential for any production use where cost controls matter - Affects safety profile enforcement (max_cost_per_plan limits) **Recommendation:** Upgrade from Priority/Backlog to Priority/High and assign to v3.1.0 as this represents missing core functionality needed for a minimal working cost management system.

freemo added this to the v3.2.0 milestone

2026-04-06 17:48:37 +00:00

freemo removed this from the v3.2.0 milestone

2026-04-06 20:39:53 +00:00

HAL9000 self-assigned this

2026-04-08 18:51:48 +00:00

HAL9000 added this to the v3.7.0 milestone

2026-04-08 18:51:48 +00:00

HAL9000 commented

2026-04-08 18:52:07 +00:00

Owner

Issue assigned to @HAL9000 and milestone set to v3.7.0.

Milestone Rationale: Plan estimation with real actor registry dispatch is part of the plan lifecycle system. v3.7.0 covers advanced plan features.

Assignment Rationale: Default assignment to HAL9000 to maintain velocity.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner

Issue assigned to @HAL9000 and milestone set to **v3.7.0**. **Milestone Rationale**: Plan estimation with real actor registry dispatch is part of the plan lifecycle system. v3.7.0 covers advanced plan features. **Assignment Rationale**: Default assignment to HAL9000 to maintain velocity. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner

HAL9000 added the

labels

2026-04-09 01:50:16 +00:00