cleveragents/cleveragents-core

Fork 3

UAT: `resolve_and_validate()` uses legacy 4-level API instead of 6-level precedence chain — spec requires full override/fallback priority support #2584

New issue

Open

opened 2026-04-03 19:00:22 +00:00 by freemo · 1 comment

freemo commented

2026-04-03 19:00:22 +00:00

Owner

Metadata

Branch: fix/exec-env-resolve-and-validate-precedence
Commit Message: fix(resource): wire resolve_and_validate() to 6-level precedence chain
Milestone: v3.7.0
Parent Epic: #398

Bug Report

What Was Tested

Code analysis of src/cleveragents/application/services/execution_environment_resolver.py and src/cleveragents/tool/runner.py.

Expected Behavior (from spec)

The specification defines a 6-level execution environment precedence chain:

Plan-level execution_environment with priority: override
Project-level execution_environment with priority: override
Nearest-ancestor devcontainer (auto-detected)
Plan-level execution_environment with priority: fallback
Project-level execution_environment with priority: fallback
Host (default)

The resolve_and_validate() method should use this full 6-level chain with override/fallback priority semantics.

Actual Behavior (from code)

The resolve_and_validate() method in ExecEnvironmentResolver calls self.resolve() — the legacy 4-level API — rather than self.resolve_with_precedence() (the new 6-level API). This means when ToolRunner.execute() calls resolve_and_validate(), it does NOT apply the override/fallback priority semantics. The 6-level chain is only available via resolve_with_precedence() and resolve_with_dag(), which are not called from resolve_and_validate().

Specifically, in execution_environment_resolver.py:

def resolve_and_validate(self, ...):
    env = self.resolve(  # ← calls legacy 4-level API, not resolve_with_precedence()
        tool_env=tool_env,
        plan_env=plan_env,
        project_env=project_env,
        default=default,
    )

The resolve() method simply picks the first non-None value from (tool_env, plan_env, project_env, default) without any priority/override/fallback semantics. This means:

A plan-level fallback environment will override a project-level override environment (wrong)
A project-level override environment will not win over a plan-level fallback environment (wrong)
The devcontainer auto-discovery (Level 3) is never consulted

Code Location

src/cleveragents/application/services/execution_environment_resolver.py — resolve_and_validate() method
src/cleveragents/tool/runner.py — execute() method calls resolve_and_validate()

Steps to Reproduce

Configure a project with execution_environment: container and priority: override
Configure a plan with execution_environment: host and priority: fallback
Execute a tool — it should use container (project override wins) but instead uses host (plan env wins because it's checked first in the legacy 4-level API)

Impact

Override/fallback priority semantics are silently ignored for all tool executions
A plan-level fallback environment incorrectly wins over a project-level override environment
The devcontainer auto-discovery (Level 3 of the precedence chain) is never consulted during tool execution
The 6-level precedence chain — a core spec requirement — is effectively dead code in the production path

Severity

High — the 6-level precedence chain is a core spec requirement for execution environment routing, and the current implementation silently ignores override/fallback priorities.

Subtasks

Update resolve_and_validate() in ExecEnvironmentResolver to call resolve_with_precedence() instead of resolve(), passing the correct priority-annotated inputs
Ensure resolve_with_precedence() signature is compatible with all call sites of resolve_and_validate() (update call sites in ToolRunner.execute() and any other callers as needed)
Verify devcontainer auto-discovery (Level 3) is correctly consulted via resolve_with_precedence() / resolve_with_dag()
Add/update BDD scenarios in Behave for resolve_and_validate() exercising all 6 precedence levels, including override/fallback semantics
Add/update integration tests (Robot Framework) for end-to-end tool execution with override and fallback priority configurations
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

resolve_and_validate() delegates to resolve_with_precedence() (or equivalent 6-level API) and correctly applies override/fallback priority semantics
A plan-level fallback environment no longer wins over a project-level override environment
Devcontainer auto-discovery (Level 3) is consulted during tool execution via resolve_and_validate()
All subtasks above are completed and checked off
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done
All nox stages pass
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/exec-env-resolve-and-validate-precedence` - **Commit Message**: `fix(resource): wire resolve_and_validate() to 6-level precedence chain` - **Milestone**: v3.7.0 - **Parent Epic**: #398 ## Bug Report ### What Was Tested Code analysis of `src/cleveragents/application/services/execution_environment_resolver.py` and `src/cleveragents/tool/runner.py`. ### Expected Behavior (from spec) The specification defines a 6-level execution environment precedence chain: 1. Plan-level `execution_environment` with `priority: override` 2. Project-level `execution_environment` with `priority: override` 3. Nearest-ancestor devcontainer (auto-detected) 4. Plan-level `execution_environment` with `priority: fallback` 5. Project-level `execution_environment` with `priority: fallback` 6. Host (default) The `resolve_and_validate()` method should use this full 6-level chain with override/fallback priority semantics. ### Actual Behavior (from code) The `resolve_and_validate()` method in `ExecEnvironmentResolver` calls `self.resolve()` — the **legacy 4-level API** — rather than `self.resolve_with_precedence()` (the new 6-level API). This means when `ToolRunner.execute()` calls `resolve_and_validate()`, it does NOT apply the override/fallback priority semantics. The 6-level chain is only available via `resolve_with_precedence()` and `resolve_with_dag()`, which are not called from `resolve_and_validate()`. Specifically, in `execution_environment_resolver.py`: ```python def resolve_and_validate(self, ...): env = self.resolve( # ← calls legacy 4-level API, not resolve_with_precedence() tool_env=tool_env, plan_env=plan_env, project_env=project_env, default=default, ) ``` The `resolve()` method simply picks the first non-None value from `(tool_env, plan_env, project_env, default)` without any priority/override/fallback semantics. This means: - A plan-level `fallback` environment will override a project-level `override` environment (wrong) - A project-level `override` environment will not win over a plan-level `fallback` environment (wrong) - The devcontainer auto-discovery (Level 3) is never consulted ### Code Location - `src/cleveragents/application/services/execution_environment_resolver.py` — `resolve_and_validate()` method - `src/cleveragents/tool/runner.py` — `execute()` method calls `resolve_and_validate()` ### Steps to Reproduce 1. Configure a project with `execution_environment: container` and `priority: override` 2. Configure a plan with `execution_environment: host` and `priority: fallback` 3. Execute a tool — it should use `container` (project override wins) but instead uses `host` (plan env wins because it's checked first in the legacy 4-level API) ### Impact - Override/fallback priority semantics are silently ignored for all tool executions - A plan-level `fallback` environment incorrectly wins over a project-level `override` environment - The devcontainer auto-discovery (Level 3 of the precedence chain) is never consulted during tool execution - The 6-level precedence chain — a core spec requirement — is effectively dead code in the production path ### Severity **High** — the 6-level precedence chain is a core spec requirement for execution environment routing, and the current implementation silently ignores override/fallback priorities. ## Subtasks - [ ] Update `resolve_and_validate()` in `ExecEnvironmentResolver` to call `resolve_with_precedence()` instead of `resolve()`, passing the correct priority-annotated inputs - [ ] Ensure `resolve_with_precedence()` signature is compatible with all call sites of `resolve_and_validate()` (update call sites in `ToolRunner.execute()` and any other callers as needed) - [ ] Verify devcontainer auto-discovery (Level 3) is correctly consulted via `resolve_with_precedence()` / `resolve_with_dag()` - [ ] Add/update BDD scenarios in Behave for `resolve_and_validate()` exercising all 6 precedence levels, including override/fallback semantics - [ ] Add/update integration tests (Robot Framework) for end-to-end tool execution with override and fallback priority configurations - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done - [ ] `resolve_and_validate()` delegates to `resolve_with_precedence()` (or equivalent 6-level API) and correctly applies override/fallback priority semantics - [ ] A plan-level `fallback` environment no longer wins over a project-level `override` environment - [ ] Devcontainer auto-discovery (Level 3) is consulted during tool execution via `resolve_and_validate()` - [ ] All subtasks above are completed and checked off - [ ] A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation - [ ] The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly - [ ] The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done - All nox stages pass - Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator

freemo added this to the v3.7.0 milestone

2026-04-03 19:00:42 +00:00

freemo added a new dependency

2026-04-03 19:03:49 +00:00

#398 Epic: Post-MVP Resources

freemo referenced this issue

2026-04-03 19:03:52 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 1591 — v3.7.0 #2593

freemo added a new dependency

2026-04-03 19:08:50 +00:00

#2597 fix(ci): restore all CI quality gates to passing on master — no suppression, no bypasses

freemo referenced this issue

2026-04-03 19:14:03 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 1601 — v3.7.0 #2603

freemo referenced this issue

2026-04-03 19:24:12 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 1611 — v3.7.0 #2605

freemo commented

2026-04-03 19:34:04 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
MoSCoW: Should Have

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: Should Have --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo referenced this issue

2026-04-03 19:34:18 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 1621 — v3.7.0 #2607

freemo referenced this issue

2026-04-03 19:44:25 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 1631 — v3.7.0 #2613

freemo referenced this issue

2026-04-03 19:54:34 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 1641 — v3.7.0 #2619

freemo referenced this issue

2026-04-03 20:04:40 +00:00

[UAT-SUPERVISOR] Progress Report — Cycle 1651 — v3.7.0 #2620