cleveragents/cleveragents-core

Fork 3

UAT: Skill and Action YAML env var interpolation silently ignores missing variables — spec requires error when `${VAR}` has no default and var is unset #4913

New issue

Open

opened 2026-04-08 20:19:12 +00:00 by HAL9000 · 2 comments

HAL9000 commented

2026-04-08 20:19:12 +00:00

Owner

Bug Report

Feature Area: Configuration System — Env Var Interpolation in Skill/Action YAML
Severity: Medium — missing env vars silently pass through as literal ${VAR} strings instead of raising an error
Found by: UAT tester, code analysis

What Was Tested

The ${ENV_VAR} interpolation behavior in SkillConfigSchema and ActionConfigSchema was compared against the spec's requirement.

Expected Behavior (from spec)

Per docs/specification.md §Design Principles (line 30983):

All configuration files support ${ENV_VAR} and ${ENV_VAR:default_value} syntax for environment variable interpolation. If a variable is not set and no default is provided, an error is raised. Boolean strings (true/false) and numeric strings are automatically converted to their native types.

Actual Behavior

SkillConfigSchema (src/cleveragents/skills/schema.py, lines 518–521):

def _env_replacer(match: re.Match[str]) -> str:
    """Replace a single ``${VAR}`` match with its env value."""
    var_name = match.group(1)
    return os.environ.get(var_name, match.group(0))  # ← returns "${VAR}" if not set!

ActionConfigSchema (src/cleveragents/action/schema.py, lines 476–479):

def _env_replacer(match: re.Match[str]) -> str:
    """Replace a single ``${VAR}`` match with its env value."""
    var_name = match.group(1)
    return os.environ.get(var_name, match.group(0))  # ← returns "${VAR}" if not set!

Both implementations silently leave the ${VAR} placeholder as-is when the environment variable is not set. This means:

A skill YAML with url: ${MY_API_URL} will load successfully even if MY_API_URL is not set
The resulting url field will contain the literal string "${MY_API_URL}" instead of raising an error
This can cause confusing runtime failures when the URL is actually used

Contrast with ActorConfiguration._interpolate_env_vars() (src/cleveragents/actor/config.py, lines 168–181), which correctly raises an error:

def replace_env_var(match: re.Match[str]) -> str:
    env_var = match.group(1)
    default_value = match.group(2) if match.group(2) is not None else None
    env_value = os.environ.get(env_var)
    if env_value is None:
        if default_value is not None:
            return str(default_value)
        raise ValueError(f"Environment variable '{env_var}' is not set")  # ← correct!
    return env_value

Code Locations

src/cleveragents/skills/schema.py, lines 518–521 (_env_replacer)
src/cleveragents/action/schema.py, lines 476–479 (_env_replacer)

Steps to Reproduce

import os
from cleveragents.skills.schema import SkillConfigSchema

# Ensure the variable is NOT set
os.environ.pop("MY_SECRET_KEY", None)

yaml_content = """
name: local/my-skill
description: "Test skill"
mcp_servers:
  - name: my-server
    transport: sse
    url: ${MY_SECRET_KEY}  # ← not set, no default
"""

# Should raise ValueError, but silently succeeds
skill = SkillConfigSchema.from_yaml(yaml_content)
print(skill.mcp_servers[0].url)  # Prints: "${MY_SECRET_KEY}" — wrong!

Impact

Skill and action YAML files with unset env vars load silently with literal ${VAR} strings
Runtime failures occur later when the placeholder string is used (e.g., as a URL or API key)
Inconsistent behavior: ActorConfiguration raises an error, but SkillConfigSchema and ActionConfigSchema don't
Users cannot detect misconfiguration at load time

Definition of Done

SkillConfigSchema._env_replacer() raises ValueError when ${VAR} is used without a default and the env var is not set
ActionConfigSchema._env_replacer() raises ValueError in the same case
${VAR:default} syntax still works (uses default when var is unset)
Error message is actionable: "Environment variable 'VAR' is not set and no default was provided"

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Bug Report **Feature Area:** Configuration System — Env Var Interpolation in Skill/Action YAML **Severity:** Medium — missing env vars silently pass through as literal `${VAR}` strings instead of raising an error **Found by:** UAT tester, code analysis --- ### What Was Tested The `${ENV_VAR}` interpolation behavior in `SkillConfigSchema` and `ActionConfigSchema` was compared against the spec's requirement. ### Expected Behavior (from spec) Per `docs/specification.md` §Design Principles (line 30983): > All configuration files support `${ENV_VAR}` and `${ENV_VAR:default_value}` syntax for environment variable interpolation. **If a variable is not set and no default is provided, an error is raised.** Boolean strings (`true`/`false`) and numeric strings are automatically converted to their native types. ### Actual Behavior **`SkillConfigSchema`** (`src/cleveragents/skills/schema.py`, lines 518–521): ```python def _env_replacer(match: re.Match[str]) -> str: """Replace a single ``${VAR}`` match with its env value.""" var_name = match.group(1) return os.environ.get(var_name, match.group(0)) # ← returns "${VAR}" if not set! ``` **`ActionConfigSchema`** (`src/cleveragents/action/schema.py`, lines 476–479): ```python def _env_replacer(match: re.Match[str]) -> str: """Replace a single ``${VAR}`` match with its env value.""" var_name = match.group(1) return os.environ.get(var_name, match.group(0)) # ← returns "${VAR}" if not set! ``` Both implementations silently leave the `${VAR}` placeholder as-is when the environment variable is not set. This means: 1. A skill YAML with `url: ${MY_API_URL}` will load successfully even if `MY_API_URL` is not set 2. The resulting `url` field will contain the literal string `"${MY_API_URL}"` instead of raising an error 3. This can cause confusing runtime failures when the URL is actually used **Contrast with `ActorConfiguration._interpolate_env_vars()`** (`src/cleveragents/actor/config.py`, lines 168–181), which correctly raises an error: ```python def replace_env_var(match: re.Match[str]) -> str: env_var = match.group(1) default_value = match.group(2) if match.group(2) is not None else None env_value = os.environ.get(env_var) if env_value is None: if default_value is not None: return str(default_value) raise ValueError(f"Environment variable '{env_var}' is not set") # ← correct! return env_value ``` ### Code Locations - `src/cleveragents/skills/schema.py`, lines 518–521 (`_env_replacer`) - `src/cleveragents/action/schema.py`, lines 476–479 (`_env_replacer`) ### Steps to Reproduce ```python import os from cleveragents.skills.schema import SkillConfigSchema # Ensure the variable is NOT set os.environ.pop("MY_SECRET_KEY", None) yaml_content = """ name: local/my-skill description: "Test skill" mcp_servers: - name: my-server transport: sse url: ${MY_SECRET_KEY} # ← not set, no default """ # Should raise ValueError, but silently succeeds skill = SkillConfigSchema.from_yaml(yaml_content) print(skill.mcp_servers[0].url) # Prints: "${MY_SECRET_KEY}" — wrong! ``` ### Impact - Skill and action YAML files with unset env vars load silently with literal `${VAR}` strings - Runtime failures occur later when the placeholder string is used (e.g., as a URL or API key) - Inconsistent behavior: `ActorConfiguration` raises an error, but `SkillConfigSchema` and `ActionConfigSchema` don't - Users cannot detect misconfiguration at load time ### Definition of Done - [ ] `SkillConfigSchema._env_replacer()` raises `ValueError` when `${VAR}` is used without a default and the env var is not set - [ ] `ActionConfigSchema._env_replacer()` raises `ValueError` in the same case - [ ] `${VAR:default}` syntax still works (uses default when var is unset) - [ ] Error message is actionable: `"Environment variable 'VAR' is not set and no default was provided"` --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 referenced this issue

2026-04-08 20:19:44 +00:00

[Automated] CleverAgents Build Session - 2026-04-08 #4799

HAL9000 referenced this issue

2026-04-08 20:22:25 +00:00

[Automated] CleverAgents Build Session - 2026-04-08 #4799

HAL9000 referenced this issue

2026-04-08 22:58:56 +00:00

Proposal: improve uat-tester — enforce label compliance when filing bug issues #4924

HAL9000 added the

labels

2026-04-08 22:59:15 +00:00

HAL9000 added this to the v3.3.0 milestone

2026-04-08 23:00:10 +00:00

HAL9000 commented

2026-04-08 23:00:30 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Medium — silent env var interpolation failure causes confusing runtime errors; inconsistent with ActorConfiguration which correctly raises errors
Milestone: v3.3.0 — Configuration system bug affecting Skill and Action YAML loading; fix is straightforward
Story Points: 2 — S — two-line fix in _env_replacer() in both skills/schema.py and action/schema.py, plus tests
MoSCoW: Should Have — spec explicitly requires error when ${VAR} has no default and var is unset; fix is low-risk and high-value for user experience
Parent Epic: Configuration System epic

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium — silent env var interpolation failure causes confusing runtime errors; inconsistent with `ActorConfiguration` which correctly raises errors - **Milestone**: v3.3.0 — Configuration system bug affecting Skill and Action YAML loading; fix is straightforward - **Story Points**: 2 — S — two-line fix in `_env_replacer()` in both `skills/schema.py` and `action/schema.py`, plus tests - **MoSCoW**: Should Have — spec explicitly requires error when `${VAR}` has no default and var is unset; fix is low-risk and high-value for user experience - **Parent Epic**: Configuration System epic --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner

HAL9000 referenced this issue

2026-04-08 23:00:50 +00:00

Proposal: update specification — env var interpolation error behavior for SkillConfigSchema and ActionConfigSchema #4934

freemo referenced this issue

2026-04-08 23:38:36 +00:00

Proposal: update specification — env var interpolation error behavior for SkillConfigSchema and ActionConfigSchema #4934

HAL9000 referenced this issue

2026-04-09 00:20:56 +00:00

Proposal: update specification — env var interpolation error behavior for SkillConfigSchema and ActionConfigSchema #4934

HAL9000 added the

Points

label

2026-04-09 05:42:35 +00:00

HAL9000 commented

2026-04-09 05:43:04 +00:00

Author

Owner

Label compliance fix applied:

Added missing label: Points/3 (M — medium complexity)
Reason: Issue is in State/Verified but was missing a story points estimate. Estimated as Points/3 (M) based on single-area bug fix with moderate complexity.

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: backlog-groomer

Label compliance fix applied: - Added missing label: `Points/3` (M — medium complexity) - Reason: Issue is in `State/Verified` but was missing a story points estimate. Estimated as Points/3 (M) based on single-area bug fix with moderate complexity. --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: backlog-groomer