feat: implement automation profile precedence chain plan action global
CI / lint (pull_request) Successful in 56s
CI / quality (pull_request) Successful in 55s
CI / typecheck (pull_request) Successful in 1m2s
CI / security (pull_request) Successful in 1m11s
CI / helm (pull_request) Successful in 31s
CI / build (pull_request) Successful in 33s
CI / push-validation (pull_request) Successful in 29s
CI / unit_tests (pull_request) Successful in 6m49s
CI / docker (pull_request) Successful in 1m32s
CI / integration_tests (pull_request) Successful in 9m20s
CI / coverage (pull_request) Successful in 8m1s
CI / status-check (pull_request) Successful in 5s
CI / lint (pull_request) Successful in 56s
CI / quality (pull_request) Successful in 55s
CI / typecheck (pull_request) Successful in 1m2s
CI / security (pull_request) Successful in 1m11s
CI / helm (pull_request) Successful in 31s
CI / build (pull_request) Successful in 33s
CI / push-validation (pull_request) Successful in 29s
CI / unit_tests (pull_request) Successful in 6m49s
CI / docker (pull_request) Successful in 1m32s
CI / integration_tests (pull_request) Successful in 9m20s
CI / coverage (pull_request) Successful in 8m1s
CI / status-check (pull_request) Successful in 5s
Implementation of the four-level automation profile precedence chain
(plan > action > project > global) as defined in the v3.5.0 specification.
Core implementation (src/):
- PrecedenceSource StrEnum with PLAN, ACTION, PROJECT, GLOBAL levels
- PrecedenceResolution frozen dataclass capturing full resolution state
- resolve_precedence_chain() with plan > action > project > global logic
- _resolve_global_profile() with explicit > env var > default fallback
- Comprehensive debug logging for observability
BDD tests (features/):
- automation_profile_precedence_chain.feature: 30 scenarios covering all
16 combinations of plan/action/project/global configurations plus edge
cases, logging verification, enum values, env var override, and custom registry
- step definitions with proper log handler cleanup via context._cleanup_handlers
CI compliance fixes (bd8b6748):
- ruff format applied to step definitions (fixes CI lint gate)
- CHANGELOG.md updated with accurate 4-level chain description
- test_reports/ artifacts removed; directory added to .gitignore
- tdd_a2a_sdk_dependency.feature reverted to use correct Client import
ISSUES CLOSED: #8234
This commit is contained in:
+2
-1
@@ -181,7 +181,8 @@ report.html
|
||||
.agent-orchestration
|
||||
agents-test
|
||||
|
||||
# Generated test reports (CI artifacts) — build artifacts, not to be committed
|
||||
|
||||
# Generated test artifacts and reports (CI/Build)
|
||||
test_reports/
|
||||
|
||||
# Auto-added by CleverAgents plan executor
|
||||
|
||||
@@ -436,6 +436,13 @@ _ALL_DATA_COLUMNS + ") " "SELECT " + _ALL_DATA_COLUMNS + " FROM v3_plans"`.
|
||||
|
||||
### Added
|
||||
|
||||
- **Automation Profile Precedence Chain** (#8234): Implemented and validated the
|
||||
four-level automation profile precedence chain (plan > action > project > global) as a
|
||||
dedicated `automation_profile_precedence` module. All 16 combinations of
|
||||
plan/action/project/global configurations are tested with BDD scenarios.
|
||||
Resolution chain is logged at debug level via the
|
||||
`PrecedenceResolution` dataclass and `PrecedenceSource` enum.
|
||||
|
||||
- `agents actor context clear` command to reset actor message history and
|
||||
state while preserving the underlying context directory via `ContextManager`
|
||||
(#6370).
|
||||
|
||||
@@ -0,0 +1,236 @@
|
||||
Feature: Automation Profile Precedence Chain (plan > action > project > global)
|
||||
The automation profile resolution follows a strict four-level precedence
|
||||
chain: plan-level > action-level > project-level > global-level. This ensures that
|
||||
fine-grained overrides work as intended for autonomous plan execution.
|
||||
|
||||
# ============================================================
|
||||
# All 16 combinations: plan(set/unset) × action(set/unset) × project(set/unset) × global(set/default)
|
||||
# ============================================================
|
||||
|
||||
# --- Combination 1: plan=set, action=set, project=set, global=set → plan wins ---
|
||||
|
||||
Scenario: Combination 1 - plan overrides action, project, and explicit global
|
||||
Given the global profile is configured as "cautious"
|
||||
When I resolve the precedence chain with plan "ci" action "auto" project "trusted"
|
||||
Then the resolved precedence profile name should be "ci"
|
||||
And the precedence source should be "plan"
|
||||
|
||||
# --- Combination 2: plan=set, action=set, project=set, global=default → plan wins ---
|
||||
|
||||
Scenario: Combination 2 - plan overrides action and project when global is default
|
||||
Given the global profile is the default
|
||||
When I resolve the precedence chain with plan "ci" action "auto" project "trusted"
|
||||
Then the resolved precedence profile name should be "ci"
|
||||
And the precedence source should be "plan"
|
||||
|
||||
# --- Combination 3: plan=set, action=set, project=unset, global=set → plan wins ---
|
||||
|
||||
Scenario: Combination 3 - plan overrides action and explicit global when project is absent
|
||||
Given the global profile is configured as "cautious"
|
||||
When I resolve the precedence chain with plan "ci" action "auto" and no project
|
||||
Then the resolved precedence profile name should be "ci"
|
||||
And the precedence source should be "plan"
|
||||
|
||||
# --- Combination 4: plan=set, action=set, project=unset, global=default → plan wins ---
|
||||
|
||||
Scenario: Combination 4 - plan overrides action when project and global are default
|
||||
Given the global profile is the default
|
||||
When I resolve the precedence chain with plan "ci" action "auto" and no project
|
||||
Then the resolved precedence profile name should be "ci"
|
||||
And the precedence source should be "plan"
|
||||
|
||||
# --- Combination 5: plan=set, action=unset, project=set, global=set → plan wins ---
|
||||
|
||||
Scenario: Combination 5 - plan overrides project and explicit global when action is absent
|
||||
Given the global profile is configured as "cautious"
|
||||
When I resolve the precedence chain with plan "ci" and no action project "trusted"
|
||||
Then the resolved precedence profile name should be "ci"
|
||||
And the precedence source should be "plan"
|
||||
|
||||
# --- Combination 6: plan=set, action=unset, project=set, global=default → plan wins ---
|
||||
|
||||
Scenario: Combination 6 - plan overrides project when action and global are default
|
||||
Given the global profile is the default
|
||||
When I resolve the precedence chain with plan "ci" and no action project "trusted"
|
||||
Then the resolved precedence profile name should be "ci"
|
||||
And the precedence source should be "plan"
|
||||
|
||||
# --- Combination 7: plan=set, action=unset, project=unset, global=set → plan wins ---
|
||||
|
||||
Scenario: Combination 7 - plan overrides explicit global when action and project are absent
|
||||
Given the global profile is configured as "cautious"
|
||||
When I resolve the precedence chain with plan "ci" and no action and no project
|
||||
Then the resolved precedence profile name should be "ci"
|
||||
And the precedence source should be "plan"
|
||||
|
||||
# --- Combination 8: plan=set, action=unset, project=unset, global=default → plan wins ---
|
||||
|
||||
Scenario: Combination 8 - plan overrides default global when action and project are absent
|
||||
Given the global profile is the default
|
||||
When I resolve the precedence chain with plan "ci" and no action and no project
|
||||
Then the resolved precedence profile name should be "ci"
|
||||
And the precedence source should be "plan"
|
||||
|
||||
# --- Combination 9: plan=unset, action=set, project=set, global=set → action wins ---
|
||||
|
||||
Scenario: Combination 9 - action overrides project and explicit global when plan is absent
|
||||
Given the global profile is configured as "cautious"
|
||||
When I resolve the precedence chain with no plan and action "auto" project "trusted"
|
||||
Then the resolved precedence profile name should be "auto"
|
||||
And the precedence source should be "action"
|
||||
|
||||
# --- Combination 10: plan=unset, action=set, project=set, global=default → action wins ---
|
||||
|
||||
Scenario: Combination 10 - action overrides project when plan and global are default
|
||||
Given the global profile is the default
|
||||
When I resolve the precedence chain with no plan and action "auto" project "trusted"
|
||||
Then the resolved precedence profile name should be "auto"
|
||||
And the precedence source should be "action"
|
||||
|
||||
# --- Combination 11: plan=unset, action=set, project=unset, global=set → action wins ---
|
||||
|
||||
Scenario: Combination 11 - action overrides explicit global when plan and project are absent
|
||||
Given the global profile is configured as "cautious"
|
||||
When I resolve the precedence chain with no plan and action "auto" and no project
|
||||
Then the resolved precedence profile name should be "auto"
|
||||
And the precedence source should be "action"
|
||||
|
||||
# --- Combination 12: plan=unset, action=set, project=unset, global=default → action wins ---
|
||||
|
||||
Scenario: Combination 12 - action overrides default global when plan and project are absent
|
||||
Given the global profile is the default
|
||||
When I resolve the precedence chain with no plan and action "auto" and no project
|
||||
Then the resolved precedence profile name should be "auto"
|
||||
And the precedence source should be "action"
|
||||
|
||||
# --- Combination 13: plan=unset, action=unset, project=set, global=set → project wins ---
|
||||
|
||||
Scenario: Combination 13 - project overrides explicit global when plan and action are absent
|
||||
Given the global profile is configured as "cautious"
|
||||
When I resolve the precedence chain with no plan and no action project "trusted"
|
||||
Then the resolved precedence profile name should be "trusted"
|
||||
And the precedence source should be "project"
|
||||
|
||||
# --- Combination 14: plan=unset, action=unset, project=set, global=default → project wins ---
|
||||
|
||||
Scenario: Combination 14 - project overrides default global when plan and action are absent
|
||||
Given the global profile is the default
|
||||
When I resolve the precedence chain with no plan and no action project "trusted"
|
||||
Then the resolved precedence profile name should be "trusted"
|
||||
And the precedence source should be "project"
|
||||
|
||||
# --- Combination 15: plan=unset, action=unset, project=unset, global=set → global wins ---
|
||||
|
||||
Scenario: Combination 15 - explicit global used when plan, action, and project are absent
|
||||
Given the global profile is configured as "cautious"
|
||||
When I resolve the precedence chain with no plan and no action and no project
|
||||
Then the resolved precedence profile name should be "cautious"
|
||||
And the precedence source should be "global"
|
||||
|
||||
# --- Combination 16: plan=unset, action=unset, project=unset, global=default → global default ---
|
||||
|
||||
Scenario: Combination 16 - default global (manual) used when nothing is set
|
||||
Given the global profile is the default
|
||||
When I resolve the precedence chain with no plan and no action and no project
|
||||
Then the resolved precedence profile name should be "manual"
|
||||
And the precedence source should be "global"
|
||||
|
||||
# ============================================================
|
||||
# Debug logging verification
|
||||
# ============================================================
|
||||
|
||||
Scenario: Resolution chain is logged at debug level
|
||||
Given the global profile is the default
|
||||
And debug logging is captured for the precedence module
|
||||
When I resolve the precedence chain with plan "ci" action "auto" project "trusted"
|
||||
Then a debug log entry should mention the resolved profile name "ci"
|
||||
And the debug log entry should mention the source "plan"
|
||||
|
||||
# ============================================================
|
||||
# Resolution result fields
|
||||
# ============================================================
|
||||
|
||||
Scenario: Resolution result exposes plan profile field
|
||||
Given the global profile is the default
|
||||
When I resolve the precedence chain with plan "ci" action "auto" project "trusted"
|
||||
Then the resolution plan_profile field should be "ci"
|
||||
And the resolution action_profile field should be "auto"
|
||||
And the resolution project_profile field should be "trusted"
|
||||
|
||||
Scenario: Resolution result exposes global profile field
|
||||
Given the global profile is configured as "cautious"
|
||||
When I resolve the precedence chain with no plan and no action and no project
|
||||
Then the resolution global_profile field should be "cautious"
|
||||
|
||||
Scenario: Resolution result exposes the full AutomationProfile object
|
||||
Given the global profile is the default
|
||||
When I resolve the precedence chain with plan "manual" and no action and no project
|
||||
Then the resolved precedence profile object should have name "manual"
|
||||
And the resolved precedence profile decompose_task should be 1.0
|
||||
|
||||
# ============================================================
|
||||
# Environment variable global override
|
||||
# ============================================================
|
||||
|
||||
Scenario: Env var sets global profile when no explicit global is configured
|
||||
Given the global profile is the default
|
||||
And the env var CLEVERAGENTS_AUTOMATION_PROFILE is "trusted"
|
||||
When I resolve the precedence chain with no plan and no action and no project
|
||||
Then the resolved precedence profile name should be "trusted"
|
||||
And the precedence source should be "global"
|
||||
|
||||
Scenario: Plan-level still overrides env var global
|
||||
Given the global profile is the default
|
||||
And the env var CLEVERAGENTS_AUTOMATION_PROFILE is "trusted"
|
||||
When I resolve the precedence chain with plan "ci" and no action and no project
|
||||
Then the resolved precedence profile name should be "ci"
|
||||
And the precedence source should be "plan"
|
||||
|
||||
# ============================================================
|
||||
# Custom profile registry
|
||||
# ============================================================
|
||||
|
||||
Scenario: Custom profile registry is used for resolution
|
||||
Given the global profile is the default
|
||||
And a custom profile registry contains "acme/deploy" with select_tool 0.5
|
||||
When I resolve the precedence chain with plan "acme/deploy" and no action and no project
|
||||
Then the resolved precedence profile name should be "acme/deploy"
|
||||
And the precedence source should be "plan"
|
||||
|
||||
# ============================================================
|
||||
# Empty string treated as absent
|
||||
# ============================================================
|
||||
|
||||
Scenario: Empty string plan profile is treated as absent
|
||||
Given the global profile is the default
|
||||
When I resolve the precedence chain with empty plan and action "auto" and no project
|
||||
Then the resolved precedence profile name should be "auto"
|
||||
And the precedence source should be "action"
|
||||
|
||||
Scenario: Empty string action profile is treated as absent
|
||||
Given the global profile is the default
|
||||
When I resolve the precedence chain with plan absent and empty action and no project
|
||||
Then the resolved precedence profile name should be "manual"
|
||||
And the precedence source should be "global"
|
||||
|
||||
Scenario: Empty string project profile is treated as absent
|
||||
Given the global profile is the default
|
||||
When I resolve the precedence chain with plan absent and action absent and empty project
|
||||
Then the resolved precedence profile name should be "manual"
|
||||
And the precedence source should be "global"
|
||||
|
||||
# ============================================================
|
||||
# PrecedenceSource enum values
|
||||
# ============================================================
|
||||
|
||||
Scenario: PrecedenceSource PLAN has value "plan"
|
||||
Then the PrecedenceSource PLAN value should be "plan"
|
||||
|
||||
Scenario: PrecedenceSource ACTION has value "action"
|
||||
Then the PrecedenceSource ACTION value should be "action"
|
||||
|
||||
Scenario: PrecedenceSource PROJECT has value "project"
|
||||
Then the PrecedenceSource PROJECT value should be "project"
|
||||
|
||||
Scenario: PrecedenceSource GLOBAL has value "global"
|
||||
Then the PrecedenceSource GLOBAL value should be "global"
|
||||
@@ -0,0 +1,443 @@
|
||||
"""Step definitions for automation profile precedence chain tests.
|
||||
|
||||
Tests the four-level precedence chain: plan > action > project > global.
|
||||
Covers all 16 combinations of (plan set/unset) x (action set/unset) x
|
||||
(project set/unset) x (global set/default).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
|
||||
from behave import given, then, when
|
||||
from behave.runner import Context
|
||||
|
||||
from cleveragents.application.services.automation_profile_precedence import (
|
||||
PrecedenceResolution,
|
||||
PrecedenceSource,
|
||||
resolve_precedence_chain,
|
||||
)
|
||||
from cleveragents.domain.models.core.automation_profile import AutomationProfile
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Constants
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_ENV_VAR = "CLEVERAGENTS_AUTOMATION_PROFILE"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _cleanup_env_var(context: Context) -> None:
|
||||
"""Remove the env var if it was set during a scenario."""
|
||||
os.environ.pop(_ENV_VAR, None)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Given steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@given('the global profile is configured as "{global_name}"')
|
||||
def step_given_global_profile_configured(context: Context, global_name: str) -> None:
|
||||
"""Set an explicit global profile override."""
|
||||
context.global_override = global_name
|
||||
# Ensure env var does not interfere
|
||||
os.environ.pop(_ENV_VAR, None)
|
||||
if not hasattr(context, "env_vars_to_clean"):
|
||||
context.env_vars_to_clean = []
|
||||
|
||||
|
||||
@given("the global profile is the default")
|
||||
def step_given_global_profile_default(context: Context) -> None:
|
||||
"""Use the default global profile (no explicit override, no env var)."""
|
||||
context.global_override = None
|
||||
os.environ.pop(_ENV_VAR, None)
|
||||
if not hasattr(context, "env_vars_to_clean"):
|
||||
context.env_vars_to_clean = []
|
||||
|
||||
|
||||
@given("debug logging is captured for the precedence module")
|
||||
def step_given_debug_logging_captured(context: Context) -> None:
|
||||
"""Install a log handler to capture debug messages from the precedence module."""
|
||||
log_records: list[logging.LogRecord] = []
|
||||
context.log_records = log_records
|
||||
|
||||
class _CapturingHandler(logging.Handler):
|
||||
def emit(self, record: logging.LogRecord) -> None:
|
||||
log_records.append(record)
|
||||
|
||||
handler = _CapturingHandler()
|
||||
handler.setLevel(logging.DEBUG)
|
||||
prec_logger = logging.getLogger(
|
||||
"cleveragents.application.services.automation_profile_precedence"
|
||||
)
|
||||
prec_logger.setLevel(logging.DEBUG)
|
||||
prec_logger.addHandler(handler)
|
||||
context.log_handler = handler
|
||||
context.prec_logger = prec_logger
|
||||
|
||||
# Register cleanup to remove the handler after the scenario
|
||||
if not hasattr(context, "_cleanup_handlers"):
|
||||
context._cleanup_handlers = []
|
||||
context._cleanup_handlers.append(lambda: prec_logger.removeHandler(handler))
|
||||
|
||||
|
||||
@given('the env var CLEVERAGENTS_AUTOMATION_PROFILE is "{value}"')
|
||||
def step_given_env_var_set(context: Context, value: str) -> None:
|
||||
"""Set the CLEVERAGENTS_AUTOMATION_PROFILE env var."""
|
||||
os.environ[_ENV_VAR] = value
|
||||
if not hasattr(context, "env_vars_to_clean"):
|
||||
context.env_vars_to_clean = []
|
||||
context.env_vars_to_clean.append(_ENV_VAR)
|
||||
|
||||
|
||||
@given('a custom profile registry contains "{name}" with select_tool {threshold:g}')
|
||||
def step_given_custom_registry(context: Context, name: str, threshold: float) -> None:
|
||||
"""Create a custom profile registry with a named profile."""
|
||||
custom_profile = AutomationProfile(name=name, select_tool=threshold)
|
||||
context.custom_registry: dict[str, AutomationProfile] = {name: custom_profile}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# When steps
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when(
|
||||
'I resolve the precedence chain with plan "{plan_name}" action "{action_name}" project "{project_name}"'
|
||||
)
|
||||
def step_when_resolve_plan_action_project(
|
||||
context: Context, plan_name: str, action_name: str, project_name: str
|
||||
) -> None:
|
||||
"""Resolve with plan, action, and project all set."""
|
||||
global_override: str | None = getattr(context, "global_override", None)
|
||||
registry: dict[str, AutomationProfile] | None = getattr(
|
||||
context, "custom_registry", None
|
||||
)
|
||||
context.resolution: PrecedenceResolution = resolve_precedence_chain(
|
||||
plan_profile=plan_name,
|
||||
action_profile=action_name,
|
||||
project_profile=project_name,
|
||||
global_override=global_override,
|
||||
profile_registry=registry,
|
||||
)
|
||||
|
||||
|
||||
@when(
|
||||
'I resolve the precedence chain with plan "{plan_name}" action "{action_name}" and no project'
|
||||
)
|
||||
def step_when_resolve_plan_and_action(
|
||||
context: Context, plan_name: str, action_name: str
|
||||
) -> None:
|
||||
"""Resolve with plan and action set, project absent."""
|
||||
global_override: str | None = getattr(context, "global_override", None)
|
||||
registry: dict[str, AutomationProfile] | None = getattr(
|
||||
context, "custom_registry", None
|
||||
)
|
||||
context.resolution: PrecedenceResolution = resolve_precedence_chain(
|
||||
plan_profile=plan_name,
|
||||
action_profile=action_name,
|
||||
project_profile=None,
|
||||
global_override=global_override,
|
||||
profile_registry=registry,
|
||||
)
|
||||
|
||||
|
||||
@when(
|
||||
'I resolve the precedence chain with plan "{plan_name}" and no action project "{project_name}"'
|
||||
)
|
||||
def step_when_resolve_plan_and_project(
|
||||
context: Context, plan_name: str, project_name: str
|
||||
) -> None:
|
||||
"""Resolve with plan and project set, action absent."""
|
||||
global_override: str | None = getattr(context, "global_override", None)
|
||||
registry: dict[str, AutomationProfile] | None = getattr(
|
||||
context, "custom_registry", None
|
||||
)
|
||||
context.resolution: PrecedenceResolution = resolve_precedence_chain(
|
||||
plan_profile=plan_name,
|
||||
action_profile=None,
|
||||
project_profile=project_name,
|
||||
global_override=global_override,
|
||||
profile_registry=registry,
|
||||
)
|
||||
|
||||
|
||||
@when(
|
||||
'I resolve the precedence chain with plan "{plan_name}" and no action and no project'
|
||||
)
|
||||
def step_when_resolve_plan_only(context: Context, plan_name: str) -> None:
|
||||
"""Resolve with plan set, action and project absent."""
|
||||
global_override: str | None = getattr(context, "global_override", None)
|
||||
registry: dict[str, AutomationProfile] | None = getattr(
|
||||
context, "custom_registry", None
|
||||
)
|
||||
context.resolution = resolve_precedence_chain(
|
||||
plan_profile=plan_name,
|
||||
action_profile=None,
|
||||
project_profile=None,
|
||||
global_override=global_override,
|
||||
profile_registry=registry,
|
||||
)
|
||||
|
||||
|
||||
@when(
|
||||
'I resolve the precedence chain with no plan and action "{action_name}" project "{project_name}"'
|
||||
)
|
||||
def step_when_resolve_action_and_project(
|
||||
context: Context, action_name: str, project_name: str
|
||||
) -> None:
|
||||
"""Resolve with action and project set, plan absent."""
|
||||
global_override: str | None = getattr(context, "global_override", None)
|
||||
registry: dict[str, AutomationProfile] | None = getattr(
|
||||
context, "custom_registry", None
|
||||
)
|
||||
context.resolution: PrecedenceResolution = resolve_precedence_chain(
|
||||
plan_profile=None,
|
||||
action_profile=action_name,
|
||||
project_profile=project_name,
|
||||
global_override=global_override,
|
||||
profile_registry=registry,
|
||||
)
|
||||
|
||||
|
||||
@when(
|
||||
'I resolve the precedence chain with no plan and action "{action_name}" and no project'
|
||||
)
|
||||
def step_when_resolve_action_only(context: Context, action_name: str) -> None:
|
||||
"""Resolve with action set, plan and project absent."""
|
||||
global_override: str | None = getattr(context, "global_override", None)
|
||||
registry: dict[str, AutomationProfile] | None = getattr(
|
||||
context, "custom_registry", None
|
||||
)
|
||||
context.resolution = resolve_precedence_chain(
|
||||
plan_profile=None,
|
||||
action_profile=action_name,
|
||||
project_profile=None,
|
||||
global_override=global_override,
|
||||
profile_registry=registry,
|
||||
)
|
||||
|
||||
|
||||
@when(
|
||||
'I resolve the precedence chain with no plan and no action project "{project_name}"'
|
||||
)
|
||||
def step_when_resolve_project_only(context: Context, project_name: str) -> None:
|
||||
"""Resolve with project set, plan and action absent."""
|
||||
global_override: str | None = getattr(context, "global_override", None)
|
||||
registry: dict[str, AutomationProfile] | None = getattr(
|
||||
context, "custom_registry", None
|
||||
)
|
||||
context.resolution = resolve_precedence_chain(
|
||||
plan_profile=None,
|
||||
action_profile=None,
|
||||
project_profile=project_name,
|
||||
global_override=global_override,
|
||||
profile_registry=registry,
|
||||
)
|
||||
|
||||
|
||||
@when("I resolve the precedence chain with no plan and no action and no project")
|
||||
def step_when_resolve_global_only(context: Context) -> None:
|
||||
"""Resolve with plan, action, and project all absent."""
|
||||
global_override: str | None = getattr(context, "global_override", None)
|
||||
registry: dict[str, AutomationProfile] | None = getattr(
|
||||
context, "custom_registry", None
|
||||
)
|
||||
context.resolution = resolve_precedence_chain(
|
||||
plan_profile=None,
|
||||
action_profile=None,
|
||||
project_profile=None,
|
||||
global_override=global_override,
|
||||
profile_registry=registry,
|
||||
)
|
||||
|
||||
|
||||
@when(
|
||||
'I resolve the precedence chain with empty plan and action "{action_name}" and no project'
|
||||
)
|
||||
def step_when_resolve_empty_plan_with_action(
|
||||
context: Context, action_name: str
|
||||
) -> None:
|
||||
"""Resolve with empty string plan (treated as absent) and action set."""
|
||||
global_override: str | None = getattr(context, "global_override", None)
|
||||
context.resolution = resolve_precedence_chain(
|
||||
plan_profile="",
|
||||
action_profile=action_name,
|
||||
project_profile=None,
|
||||
global_override=global_override,
|
||||
)
|
||||
|
||||
|
||||
@when("I resolve the precedence chain with plan absent and empty action and no project")
|
||||
def step_when_resolve_absent_plan_empty_action(context: Context) -> None:
|
||||
"""Resolve with plan absent and empty string action (treated as absent)."""
|
||||
global_override: str | None = getattr(context, "global_override", None)
|
||||
context.resolution = resolve_precedence_chain(
|
||||
plan_profile=None,
|
||||
action_profile="",
|
||||
project_profile=None,
|
||||
global_override=global_override,
|
||||
)
|
||||
|
||||
|
||||
@when(
|
||||
"I resolve the precedence chain with plan absent and action absent and empty project"
|
||||
)
|
||||
def step_when_resolve_absent_plan_action_empty_project(context: Context) -> None:
|
||||
"""Resolve with plan and action absent and empty string project (treated as absent)."""
|
||||
global_override: str | None = getattr(context, "global_override", None)
|
||||
context.resolution = resolve_precedence_chain(
|
||||
plan_profile=None,
|
||||
action_profile=None,
|
||||
project_profile="",
|
||||
global_override=global_override,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then steps: resolution result
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then('the resolved precedence profile name should be "{expected}"')
|
||||
def step_then_resolved_name(context: Context, expected: str) -> None:
|
||||
"""Assert the resolved profile name."""
|
||||
actual = context.resolution.profile_name
|
||||
assert actual == expected, (
|
||||
f"Expected resolved profile name '{expected}', got '{actual}'"
|
||||
)
|
||||
|
||||
|
||||
@then('the precedence source should be "{expected_source}"')
|
||||
def step_then_precedence_source(context: Context, expected_source: str) -> None:
|
||||
"""Assert the precedence source (plan, action, project, or global)."""
|
||||
actual = context.resolution.source.value
|
||||
assert actual == expected_source, (
|
||||
f"Expected precedence source '{expected_source}', got '{actual}'"
|
||||
)
|
||||
|
||||
|
||||
@then('the resolution plan_profile field should be "{expected}"')
|
||||
def step_then_resolution_plan_field(context: Context, expected: str) -> None:
|
||||
"""Assert the plan_profile field on the resolution result."""
|
||||
actual = context.resolution.plan_profile
|
||||
assert actual == expected, (
|
||||
f"Expected resolution.plan_profile '{expected}', got '{actual}'"
|
||||
)
|
||||
|
||||
|
||||
@then('the resolution action_profile field should be "{expected}"')
|
||||
def step_then_resolution_action_field(context: Context, expected: str) -> None:
|
||||
"""Assert the action_profile field on the resolution result."""
|
||||
actual = context.resolution.action_profile
|
||||
assert actual == expected, (
|
||||
f"Expected resolution.action_profile '{expected}', got '{actual}'"
|
||||
)
|
||||
|
||||
|
||||
@then('the resolution project_profile field should be "{expected}"')
|
||||
def step_then_resolution_project_field(context: Context, expected: str) -> None:
|
||||
"""Assert the project_profile field on the resolution result."""
|
||||
actual = context.resolution.project_profile
|
||||
assert actual == expected, (
|
||||
f"Expected resolution.project_profile '{expected}', got '{actual}'"
|
||||
)
|
||||
|
||||
|
||||
@then('the resolution global_profile field should be "{expected}"')
|
||||
def step_then_resolution_global_field(context: Context, expected: str) -> None:
|
||||
"""Assert the global_profile field on the resolution result."""
|
||||
actual = context.resolution.global_profile
|
||||
assert actual == expected, (
|
||||
f"Expected resolution.global_profile '{expected}', got '{actual}'"
|
||||
)
|
||||
|
||||
|
||||
@then('the resolved precedence profile object should have name "{expected}"')
|
||||
def step_then_resolved_profile_object_name(context: Context, expected: str) -> None:
|
||||
"""Assert the resolved AutomationProfile object's name."""
|
||||
actual = context.resolution.profile.name
|
||||
assert actual == expected, f"Expected profile.name '{expected}', got '{actual}'"
|
||||
|
||||
|
||||
@then("the resolved precedence profile decompose_task should be {expected:g}")
|
||||
def step_then_resolved_profile_decompose_task(
|
||||
context: Context, expected: float
|
||||
) -> None:
|
||||
"""Assert the resolved profile's decompose_task threshold."""
|
||||
actual = context.resolution.profile.decompose_task
|
||||
assert actual == expected, (
|
||||
f"Expected profile.decompose_task {expected}, got {actual}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then steps: debug logging
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then('a debug log entry should mention the resolved profile name "{name}"')
|
||||
def step_then_debug_log_mentions_name(context: Context, name: str) -> None:
|
||||
"""Assert that a debug log entry contains the resolved profile name."""
|
||||
records: list[logging.LogRecord] = getattr(context, "log_records", [])
|
||||
messages = [r.getMessage() for r in records if r.levelno == logging.DEBUG]
|
||||
found = any(name in msg for msg in messages)
|
||||
assert found, f"Expected debug log to mention '{name}'. Debug messages: {messages}"
|
||||
|
||||
|
||||
@then('the debug log entry should mention the source "{source}"')
|
||||
def step_then_debug_log_mentions_source(context: Context, source: str) -> None:
|
||||
"""Assert that a debug log entry contains the source level."""
|
||||
records: list[logging.LogRecord] = getattr(context, "log_records", [])
|
||||
messages = [r.getMessage() for r in records if r.levelno == logging.DEBUG]
|
||||
found = any(source in msg for msg in messages)
|
||||
assert found, (
|
||||
f"Expected debug log to mention source '{source}'. Debug messages: {messages}"
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Then steps: PrecedenceSource enum
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@then('the PrecedenceSource PLAN value should be "{expected}"')
|
||||
def step_then_precedence_source_plan_value(context: Context, expected: str) -> None:
|
||||
"""Assert PrecedenceSource.PLAN has the expected string value."""
|
||||
assert PrecedenceSource.PLAN.value == expected, (
|
||||
f"Expected PrecedenceSource.PLAN.value '{expected}', "
|
||||
f"got '{PrecedenceSource.PLAN.value}'"
|
||||
)
|
||||
|
||||
|
||||
@then('the PrecedenceSource ACTION value should be "{expected}"')
|
||||
def step_then_precedence_source_action_value(context: Context, expected: str) -> None:
|
||||
"""Assert PrecedenceSource.ACTION has the expected string value."""
|
||||
assert PrecedenceSource.ACTION.value == expected, (
|
||||
f"Expected PrecedenceSource.ACTION.value '{expected}', "
|
||||
f"got '{PrecedenceSource.ACTION.value}'"
|
||||
)
|
||||
|
||||
|
||||
@then('the PrecedenceSource PROJECT value should be "{expected}"')
|
||||
def step_then_precedence_source_project_value(context: Context, expected: str) -> None:
|
||||
"""Assert PrecedenceSource.PROJECT has the expected string value."""
|
||||
assert PrecedenceSource.PROJECT.value == expected, (
|
||||
f"Expected PrecedenceSource.PROJECT.value '{expected}', "
|
||||
f"got '{PrecedenceSource.PROJECT.value}'"
|
||||
)
|
||||
|
||||
|
||||
@then('the PrecedenceSource GLOBAL value should be "{expected}"')
|
||||
def step_then_precedence_source_global_value(context: Context, expected: str) -> None:
|
||||
"""Assert PrecedenceSource.GLOBAL has the expected string value."""
|
||||
assert PrecedenceSource.GLOBAL.value == expected, (
|
||||
f"Expected PrecedenceSource.GLOBAL.value '{expected}', "
|
||||
f"got '{PrecedenceSource.GLOBAL.value}'"
|
||||
)
|
||||
@@ -0,0 +1,258 @@
|
||||
"""Automation Profile Precedence Chain for CleverAgents v3.5.
|
||||
|
||||
Implements and validates the four-level precedence chain for automation
|
||||
profile resolution:
|
||||
|
||||
plan > action > project > global
|
||||
|
||||
This module provides a dedicated, testable implementation of the precedence
|
||||
chain that is independent of the full ``AutomationProfileService`` CRUD
|
||||
operations. It is the canonical source of truth for precedence resolution
|
||||
in plan execution contexts.
|
||||
|
||||
Precedence Rules
|
||||
----------------
|
||||
|
||||
1. **Plan-level** — explicitly set via ``--automation-profile`` on plan use.
|
||||
Highest priority; overrides action, project, and global.
|
||||
2. **Action-level** — default profile set on the action template.
|
||||
Overrides project and global; used when no plan-level override is set.
|
||||
3. **Project-level** — default profile set at the project level.
|
||||
Overrides global; used when neither plan nor action override is set.
|
||||
4. **Global-level** — set via config key or
|
||||
``CLEVERAGENTS_AUTOMATION_PROFILE`` env var, falling back to ``"manual"``.
|
||||
Used when none of plan, action, or project override is set.
|
||||
|
||||
All 16 Combinations
|
||||
-------------------
|
||||
|
||||
The precedence chain covers all 16 combinations of:
|
||||
(plan set/unset) x (action set/unset) x (project set/unset) x (global set/default)
|
||||
|
||||
+-------+--------+---------+--------+------------------+
|
||||
| plan | action | project | global | resolved source |
|
||||
+=======+========+=========+========+==================+
|
||||
| set | set | set | set | plan |
|
||||
+-------+--------+---------+--------+------------------+
|
||||
| set | set | set | default| plan |
|
||||
+-------+--------+---------+--------+------------------+
|
||||
| set | set | unset | set | plan |
|
||||
+-------+--------+---------+--------+------------------+
|
||||
| set | set | unset | default| plan |
|
||||
+-------+--------+---------+--------+------------------+
|
||||
| set | unset | set | set | plan |
|
||||
+-------+--------+---------+--------+------------------+
|
||||
| set | unset | set | default| plan |
|
||||
+-------+--------+---------+--------+------------------+
|
||||
| set | unset | unset | set | plan |
|
||||
+-------+--------+---------+--------+------------------+
|
||||
| set | unset | unset | default| plan |
|
||||
+-------+--------+---------+--------+------------------+
|
||||
| unset | set | set | set | action |
|
||||
+-------+--------+---------+--------+------------------+
|
||||
| unset | set | set | default| action |
|
||||
+-------+--------+---------+--------+------------------+
|
||||
| unset | set | unset | set | action |
|
||||
+-------+--------+---------+--------+------------------+
|
||||
| unset | set | unset | default| action |
|
||||
+-------+--------+---------+--------+------------------+
|
||||
| unset | unset | set | set | project |
|
||||
+-------+--------+---------+--------+------------------+
|
||||
| unset | unset | set | default| project |
|
||||
+-------+--------+---------+--------+------------------+
|
||||
| unset | unset | unset | set | global |
|
||||
+-------+--------+---------+--------+------------------+
|
||||
| unset | unset | unset | default| global (manual) |
|
||||
+-------+--------+---------+--------+------------------+
|
||||
|
||||
Based on ``docs/specification.md`` Section "Automation Profiles".
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
from dataclasses import dataclass
|
||||
from enum import StrEnum
|
||||
|
||||
from cleveragents.domain.models.core.automation_profile import (
|
||||
BUILTIN_PROFILES,
|
||||
AutomationProfile,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_ENV_VAR = "CLEVERAGENTS_AUTOMATION_PROFILE"
|
||||
_DEFAULT_GLOBAL_PROFILE = "manual"
|
||||
|
||||
|
||||
class PrecedenceSource(StrEnum):
|
||||
"""Indicates which level of the precedence chain resolved the profile.
|
||||
|
||||
Values correspond to the four levels of the plan > action > project > global
|
||||
chain.
|
||||
"""
|
||||
|
||||
PLAN = "plan"
|
||||
ACTION = "action"
|
||||
PROJECT = "project"
|
||||
GLOBAL = "global"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class PrecedenceResolution:
|
||||
"""Result of a precedence chain resolution.
|
||||
|
||||
Attributes:
|
||||
profile: The resolved ``AutomationProfile``.
|
||||
source: Which level of the chain provided the profile.
|
||||
profile_name: The resolved profile name string.
|
||||
plan_profile: The plan-level override (may be ``None``).
|
||||
action_profile: The action-level override (may be ``None``).
|
||||
project_profile: The project-level override (may be ``None``).
|
||||
global_profile: The effective global profile name used as fallback.
|
||||
"""
|
||||
|
||||
profile: AutomationProfile
|
||||
source: PrecedenceSource
|
||||
profile_name: str
|
||||
plan_profile: str | None
|
||||
action_profile: str | None
|
||||
project_profile: str | None
|
||||
global_profile: str
|
||||
|
||||
|
||||
def _resolve_global_profile(
|
||||
global_override: str | None,
|
||||
) -> str:
|
||||
"""Resolve the effective global profile name.
|
||||
|
||||
Precedence within global resolution:
|
||||
1. Explicit ``global_override`` argument (from config service).
|
||||
2. ``CLEVERAGENTS_AUTOMATION_PROFILE`` environment variable.
|
||||
3. Hard-coded default ``"manual"``.
|
||||
|
||||
Args:
|
||||
global_override: Explicit global profile name from config, or
|
||||
``None`` to fall back to env var / default.
|
||||
|
||||
Returns:
|
||||
The effective global profile name (always a non-empty string).
|
||||
"""
|
||||
if global_override and global_override.strip():
|
||||
return global_override.strip()
|
||||
env_val = os.environ.get(_ENV_VAR, "").strip()
|
||||
if env_val:
|
||||
return env_val
|
||||
return _DEFAULT_GLOBAL_PROFILE
|
||||
|
||||
|
||||
def resolve_precedence_chain(
|
||||
*,
|
||||
plan_profile: str | None = None,
|
||||
action_profile: str | None = None,
|
||||
project_profile: str | None = None,
|
||||
global_override: str | None = None,
|
||||
profile_registry: dict[str, AutomationProfile] | None = None,
|
||||
) -> PrecedenceResolution:
|
||||
"""Resolve the effective automation profile using the precedence chain.
|
||||
|
||||
Implements the canonical plan > action > project > global precedence chain.
|
||||
The resolution is logged at ``DEBUG`` level so callers can trace
|
||||
which level provided the profile.
|
||||
|
||||
Args:
|
||||
plan_profile: Profile name set at plan level (highest priority).
|
||||
``None`` or empty string means no plan-level override.
|
||||
action_profile: Profile name set on the action template.
|
||||
``None`` or empty string means no action-level override.
|
||||
project_profile: Profile name set at the project level.
|
||||
``None`` or empty string means no project-level override.
|
||||
global_override: Explicit global profile name from config service.
|
||||
Falls back to ``CLEVERAGENTS_AUTOMATION_PROFILE`` env var,
|
||||
then ``"manual"``.
|
||||
profile_registry: Optional mapping of profile name → profile for
|
||||
custom profiles. Built-in profiles are always available.
|
||||
Defaults to ``BUILTIN_PROFILES`` only.
|
||||
|
||||
Returns:
|
||||
A :class:`PrecedenceResolution` describing the resolved profile
|
||||
and which level of the chain provided it.
|
||||
|
||||
Raises:
|
||||
KeyError: If the resolved profile name is not found in the
|
||||
built-in profiles or the provided ``profile_registry``.
|
||||
|
||||
Examples:
|
||||
>>> result = resolve_precedence_chain(plan_profile="ci")
|
||||
>>> result.source
|
||||
<PrecedenceSource.PLAN: 'plan'>
|
||||
>>> result.profile_name
|
||||
'ci'
|
||||
|
||||
>>> result = resolve_precedence_chain(action_profile="auto")
|
||||
>>> result.source
|
||||
<PrecedenceSource.ACTION: 'action'>
|
||||
|
||||
>>> result = resolve_precedence_chain(project_profile="trusted")
|
||||
>>> result.source
|
||||
<PrecedenceSource.PROJECT: 'project'>
|
||||
|
||||
>>> result = resolve_precedence_chain()
|
||||
>>> result.source
|
||||
<PrecedenceSource.GLOBAL: 'global'>
|
||||
>>> result.profile_name
|
||||
'manual'
|
||||
"""
|
||||
registry: dict[str, AutomationProfile] = dict(BUILTIN_PROFILES)
|
||||
if profile_registry:
|
||||
registry.update(profile_registry)
|
||||
|
||||
# Normalise inputs — treat empty strings as absent
|
||||
norm_plan = plan_profile.strip() if isinstance(plan_profile, str) else None
|
||||
norm_plan = norm_plan if norm_plan else None
|
||||
|
||||
norm_action = action_profile.strip() if isinstance(action_profile, str) else None
|
||||
norm_action = norm_action if norm_action else None
|
||||
|
||||
norm_project = project_profile.strip() if isinstance(project_profile, str) else None
|
||||
norm_project = norm_project if norm_project else None
|
||||
|
||||
effective_global = _resolve_global_profile(global_override)
|
||||
|
||||
# Apply precedence chain: plan > action > project > global
|
||||
if norm_plan is not None:
|
||||
chosen_name = norm_plan
|
||||
source = PrecedenceSource.PLAN
|
||||
elif norm_action is not None:
|
||||
chosen_name = norm_action
|
||||
source = PrecedenceSource.ACTION
|
||||
elif norm_project is not None:
|
||||
chosen_name = norm_project
|
||||
source = PrecedenceSource.PROJECT
|
||||
else:
|
||||
chosen_name = effective_global
|
||||
source = PrecedenceSource.GLOBAL
|
||||
|
||||
logger.debug(
|
||||
"Automation profile precedence chain resolved: "
|
||||
"name=%r source=%s plan=%r action=%r project=%r global=%r",
|
||||
chosen_name,
|
||||
source.value,
|
||||
norm_plan,
|
||||
norm_action,
|
||||
norm_project,
|
||||
effective_global,
|
||||
)
|
||||
|
||||
profile = registry[chosen_name]
|
||||
|
||||
return PrecedenceResolution(
|
||||
profile=profile,
|
||||
source=source,
|
||||
profile_name=chosen_name,
|
||||
plan_profile=norm_plan,
|
||||
action_profile=norm_action,
|
||||
project_profile=norm_project,
|
||||
global_profile=effective_global,
|
||||
)
|
||||
Reference in New Issue
Block a user