Compare commits
1 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 40b38b010b |
@@ -300,6 +300,12 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
MCP logger thread-safety in `session.py` using a threading lock. Created
|
||||
migration guide for users transitioning from legacy to V3 workflow.
|
||||
|
||||
- **Default Automation Profile** (#9152): Changed `_DEFAULT_PROFILE` in
|
||||
`AutomationProfileService` from `"manual"` to `"supervised"` to align with ADR-017.
|
||||
Updated reference documentation and BDD scenarios to reflect the new default. Also
|
||||
fixed env var security filtering in action schema (`src/cleveragents/action/schema.py`)
|
||||
to restrict interpolation to `CLEVERAGENTS_`-prefixed variables only.
|
||||
|
||||
- **Automation Profile Silent Fallback** (#8232): `_resolve_profile_for_plan` in
|
||||
`PlanLifecycleService` now raises a clear `ValidationError` when a plan's
|
||||
automation profile name is not a known built-in profile, instead of silently
|
||||
|
||||
@@ -31,3 +31,4 @@ Below are some of the specific details of various contributions.
|
||||
* HAL 9000 has contributed comprehensive milestone documentation for v3.6.0 (Advanced Concepts & Deferred Features) and v3.7.0 (TUI Implementation) (PR #9903): split into sub-documents covering context strategies, LLM backends, resource types, A2A rename, container tool execution, scope chain resolution, cost/safety budgets, E2E workflow tests, code review examples, plugin architecture, TUI layout, persona system, reference/command input, session management, configuration, and TuiMaterializer integration.
|
||||
* HAL 9000 has contributed the LLMTraceRepository data-integrity fix (PR #8185 / issue #7505): replaced the unconditional `session.commit()` in `LLMTraceRepository.save()` with a dual-path implementation that respects the UnitOfWork pattern — flushing only when an external session is provided, and flushing + committing + closing when operating standalone. This eliminates premature transaction commits, loss of rollback capability, and a docstring/implementation mismatch.
|
||||
* HAL 9000 has contributed the ACMS Index Data Model and File Traversal Engine (PR #9664 / issue #9579): foundational data structures for indexed context entries with hot/warm/cold/archive storage tier classification, tag system, and a timeout-safe chunked file traversal engine for large projects with 10,000+ files.
|
||||
* HAL 9000 has contributed the default automation profile fix (#9152): aligned `_DEFAULT_PROFILE` in `AutomationProfileService` with ADR-017 by changing the default from `"manual"` to `"supervised"`, updated reference documentation, and fixed env var security filtering in action schema.
|
||||
|
||||
@@ -16,7 +16,7 @@ Profiles are resolved in order (first non-null wins):
|
||||
| 3 | Project-level | Set via project configuration |
|
||||
| 4 | Global-level | Config key or `CLEVERAGENTS_AUTOMATION_PROFILE` env var |
|
||||
|
||||
If no profile is set at any level, the global default is `manual`.
|
||||
If no profile is set at any level, the global default is `supervised`.
|
||||
|
||||
## Configuration
|
||||
|
||||
|
||||
@@ -92,7 +92,7 @@ When determining which profile applies to a given plan execution, the system res
|
||||
1. **Plan-level override** — A profile specified directly on the plan.
|
||||
2. **Project-level setting** — The default profile configured for the project.
|
||||
3. **Organization-level default** — The organization's default profile.
|
||||
4. **System default** — Falls back to the `review` built-in profile.
|
||||
4. **System default** — Falls back to the `supervised` built-in profile.
|
||||
|
||||
At each level, the profile may be specified by:
|
||||
- A built-in name (e.g. `cautious`)
|
||||
|
||||
@@ -0,0 +1,118 @@
|
||||
@security @action-schema @env-var
|
||||
Feature: Action Schema Environment Variable Security
|
||||
As a security-conscious developer
|
||||
I want to ensure that action YAML files can only interpolate allowlisted environment variables
|
||||
So that sensitive credentials cannot be exfiltrated through malicious action configurations
|
||||
|
||||
Background:
|
||||
Given an action YAML string with only required fields for security tests
|
||||
|
||||
# ============================================================
|
||||
# Allowed Environment Variable Interpolation
|
||||
# ============================================================
|
||||
|
||||
Scenario: Interpolate CLEVERAGENTS_ prefixed environment variable
|
||||
Given an environment variable "CLEVERAGENTS_API_KEY" set to "secret-key-123"
|
||||
And an action YAML string with description "${CLEVERAGENTS_API_KEY}"
|
||||
When I load the action YAML
|
||||
Then the loaded action description should be "secret-key-123"
|
||||
|
||||
Scenario: Interpolate multiple CLEVERAGENTS_ prefixed variables
|
||||
Given an environment variable "CLEVERAGENTS_HOST" set to "localhost"
|
||||
And an environment variable "CLEVERAGENTS_PORT" set to "8080"
|
||||
And an action YAML string with description "Connect to ${CLEVERAGENTS_HOST}:${CLEVERAGENTS_PORT}"
|
||||
When I load the action YAML
|
||||
Then the loaded action description should be "Connect to localhost:8080"
|
||||
|
||||
Scenario: Missing CLEVERAGENTS_ variable left as placeholder
|
||||
Given no environment variable "CLEVERAGENTS_MISSING_VAR"
|
||||
And an action YAML string with description "Config: ${CLEVERAGENTS_MISSING_VAR}"
|
||||
When I load the action YAML
|
||||
Then the loaded action description should be "Config: ${CLEVERAGENTS_MISSING_VAR}"
|
||||
|
||||
# ============================================================
|
||||
# Disallowed Environment Variable Protection
|
||||
# ============================================================
|
||||
|
||||
Scenario: Disallow AWS_SECRET_ACCESS_KEY interpolation
|
||||
Given an environment variable "AWS_SECRET_ACCESS_KEY" set to "aws-secret-123"
|
||||
And an action YAML string with description "Secret: ${AWS_SECRET_ACCESS_KEY}"
|
||||
When I load the action YAML
|
||||
Then the loaded action description should be "Secret: ${AWS_SECRET_ACCESS_KEY}"
|
||||
And the action description should NOT contain "aws-secret-123"
|
||||
|
||||
Scenario: Disallow DATABASE_PASSWORD interpolation
|
||||
Given an environment variable "DATABASE_PASSWORD" set to "db-password-456"
|
||||
And an action YAML string with description "DB: ${DATABASE_PASSWORD}"
|
||||
When I load the action YAML
|
||||
Then the loaded action description should be "DB: ${DATABASE_PASSWORD}"
|
||||
And the action description should NOT contain "db-password-456"
|
||||
|
||||
Scenario: Disallow API_KEY interpolation
|
||||
Given an environment variable "API_KEY" set to "api-key-789"
|
||||
And an action YAML string with description "API: ${API_KEY}"
|
||||
When I load the action YAML
|
||||
Then the loaded action description should be "API: ${API_KEY}"
|
||||
And the action description should NOT contain "api-key-789"
|
||||
|
||||
Scenario: Disallow HOME interpolation
|
||||
Given an environment variable "HOME" set to "/home/user"
|
||||
And an action YAML string with description "Home: ${HOME}"
|
||||
When I load the action YAML
|
||||
Then the loaded action description should be "Home: ${HOME}"
|
||||
And the action description should NOT contain "/home/user"
|
||||
|
||||
Scenario: Disallow PATH interpolation
|
||||
Given an environment variable "PATH" set to "/usr/bin:/bin"
|
||||
And an action YAML string with description "Path: ${PATH}"
|
||||
When I load the action YAML
|
||||
Then the loaded action description should be "Path: ${PATH}"
|
||||
And the action description should NOT contain "/usr/bin"
|
||||
|
||||
# ============================================================
|
||||
# Mixed Allowed and Disallowed Variables
|
||||
# ============================================================
|
||||
|
||||
Scenario: Mix allowed and disallowed variables
|
||||
Given an environment variable "CLEVERAGENTS_CONFIG" set to "allowed-config"
|
||||
And an environment variable "SECRET_TOKEN" set to "secret-token-xyz"
|
||||
And an action YAML string with description "Config: ${CLEVERAGENTS_CONFIG}, Token: ${SECRET_TOKEN}"
|
||||
When I load the action YAML
|
||||
Then the loaded action description should be "Config: allowed-config, Token: ${SECRET_TOKEN}"
|
||||
And the action description should NOT contain "secret-token-xyz"
|
||||
|
||||
# ============================================================
|
||||
# Edge Cases
|
||||
# ============================================================
|
||||
|
||||
Scenario: Empty prefix check (no variables match)
|
||||
Given an environment variable "CLEVERAGENTS_" set to "empty-prefix"
|
||||
And an action YAML string with description "Var: ${CLEVERAGENTS_}"
|
||||
When I load the action YAML
|
||||
Then the loaded action description should be "Var: ${CLEVERAGENTS_}"
|
||||
|
||||
Scenario: Case sensitivity - lowercase not allowed
|
||||
Given an environment variable "cleveragents_api_key" set to "lowercase-key"
|
||||
And an action YAML string with description "Key: ${cleveragents_api_key}"
|
||||
When I load the action YAML
|
||||
Then the loaded action description should be "Key: ${cleveragents_api_key}"
|
||||
And the action description should NOT contain "lowercase-key"
|
||||
|
||||
Scenario: Interpolation in nested fields
|
||||
Given an environment variable "CLEVERAGENTS_ACTOR" set to "openai/gpt-4"
|
||||
And an action YAML string with strategy_actor "${CLEVERAGENTS_ACTOR}"
|
||||
When I load the action YAML
|
||||
Then the loaded action strategy_actor should be "openai/gpt-4"
|
||||
|
||||
Scenario: Interpolation in list fields
|
||||
Given an environment variable "CLEVERAGENTS_INVARIANT" set to "No secrets in code"
|
||||
And an action YAML string with invariants containing "${CLEVERAGENTS_INVARIANT}"
|
||||
When I load the action YAML
|
||||
Then the action invariants should contain "No secrets in code"
|
||||
|
||||
Scenario: Disallowed variable in list fields
|
||||
Given an environment variable "SECRET_INVARIANT" set to "Secret invariant"
|
||||
And an action YAML string with invariants containing "${SECRET_INVARIANT}"
|
||||
When I load the action YAML
|
||||
Then the action invariants should contain "${SECRET_INVARIANT}"
|
||||
And the action invariants should NOT contain "Secret invariant"
|
||||
@@ -482,10 +482,10 @@ Feature: Consolidated Automation Profile
|
||||
Then the resolved profile name should be "cautious"
|
||||
|
||||
|
||||
Scenario: Default global is manual when nothing configured
|
||||
Scenario: Default global is supervised when nothing configured
|
||||
Given an automation profile service with no configuration
|
||||
When I resolve profile with plan None action None project None
|
||||
Then the resolved profile name should be "manual"
|
||||
Then the resolved profile name should be "supervised"
|
||||
|
||||
# ---- Missing profile errors ----
|
||||
|
||||
|
||||
@@ -698,6 +698,12 @@ def after_scenario(context, scenario):
|
||||
os.environ.pop(key, None)
|
||||
context.env_vars_to_clean = []
|
||||
|
||||
# Clean up environment variables set by action_schema_env_var_security tests
|
||||
if hasattr(context, "env_vars_to_cleanup"):
|
||||
for var_name in context.env_vars_to_cleanup:
|
||||
if var_name in os.environ:
|
||||
del os.environ[var_name]
|
||||
|
||||
for env_var in [
|
||||
*LANGSMITH_ENV_VARS,
|
||||
"CLEVERAGENTS_DATABASE_URL",
|
||||
|
||||
@@ -0,0 +1,178 @@
|
||||
"""Step definitions for action schema environment variable security tests.
|
||||
|
||||
Tests for features/action_schema_env_var_security.feature — validates that
|
||||
the ActionConfigSchema only allows interpolation of CLEVERAGENTS_ prefixed
|
||||
environment variables, preventing exfiltration of sensitive credentials.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
from typing import Any
|
||||
|
||||
from behave import given, then, when # type: ignore[import-untyped]
|
||||
from behave.runner import Context # type: ignore[import-untyped]
|
||||
|
||||
from cleveragents.action.schema import ActionConfigSchema
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# Helpers
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
_MINIMAL_YAML = """\
|
||||
name: local/simple-action
|
||||
description: A simple action for testing
|
||||
strategy_actor: openai/gpt-4
|
||||
execution_actor: openai/gpt-4
|
||||
definition_of_done: All tests pass
|
||||
"""
|
||||
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# Given steps
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@given("an action YAML string with only required fields for security tests")
|
||||
def step_given_minimal_yaml(context: Context) -> None:
|
||||
"""Provide a minimal valid action YAML string."""
|
||||
context.action_yaml_string = _MINIMAL_YAML
|
||||
|
||||
|
||||
@given('an environment variable "{var_name}" set to "{var_value}"')
|
||||
def step_given_env_var_set(context: Context, var_name: str, var_value: str) -> None:
|
||||
"""Set an environment variable for testing."""
|
||||
os.environ[var_name] = var_value
|
||||
# Store for cleanup
|
||||
if not hasattr(context, "env_vars_to_cleanup"):
|
||||
context.env_vars_to_cleanup = []
|
||||
context.env_vars_to_cleanup.append(var_name)
|
||||
|
||||
|
||||
@given('an environment variable "{var_name}" set "{var_value}"')
|
||||
def step_given_env_var_set_alt(context: Context, var_name: str, var_value: str) -> None:
|
||||
"""Set an environment variable for testing (alternative syntax)."""
|
||||
os.environ[var_name] = var_value
|
||||
# Store for cleanup
|
||||
if not hasattr(context, "env_vars_to_cleanup"):
|
||||
context.env_vars_to_cleanup = []
|
||||
context.env_vars_to_cleanup.append(var_name)
|
||||
|
||||
|
||||
@given('no environment variable "{var_name}"')
|
||||
def step_given_no_env_var(context: Context, var_name: str) -> None:
|
||||
"""Ensure an environment variable is not set."""
|
||||
if var_name in os.environ:
|
||||
del os.environ[var_name]
|
||||
|
||||
|
||||
@given('an action YAML string with description "{description}"')
|
||||
def step_given_yaml_with_description(context: Context, description: str) -> None:
|
||||
"""Provide YAML with a custom description value."""
|
||||
# Use a YAML template with quoted description to handle special chars like ':'
|
||||
yaml_template = _MINIMAL_YAML.replace(
|
||||
"description: A simple action for testing", "description: PLACEHOLDER"
|
||||
)
|
||||
context.action_yaml_string = yaml_template.replace(
|
||||
"description: PLACEHOLDER", f"description: '{description}'"
|
||||
)
|
||||
|
||||
|
||||
@given('an action YAML string with strategy_actor "{actor}"')
|
||||
def step_given_yaml_with_strategy_actor(context: Context, actor: str) -> None:
|
||||
"""Provide YAML with a custom strategy_actor value."""
|
||||
context.action_yaml_string = _MINIMAL_YAML.replace("openai/gpt-4", actor, 1)
|
||||
|
||||
|
||||
@given('an action YAML string with invariants containing "{invariant}"')
|
||||
def step_given_yaml_with_invariants(context: Context, invariant: str) -> None:
|
||||
"""Provide YAML with invariants containing a specific value."""
|
||||
yaml_with_invariants = _MINIMAL_YAML + f"\ninvariants:\n - {invariant}\n"
|
||||
context.action_yaml_string = yaml_with_invariants
|
||||
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# When steps
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@when("I load the action YAML")
|
||||
def step_when_load_action_yaml(context: Context) -> None:
|
||||
"""Load and validate the action YAML."""
|
||||
try:
|
||||
context.action_config = ActionConfigSchema.from_yaml(context.action_yaml_string)
|
||||
context.load_error = None
|
||||
except Exception as e:
|
||||
context.action_config = None
|
||||
context.load_error = e
|
||||
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# Then steps
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@then('the loaded action description should be "{expected_description}"')
|
||||
def step_then_description_is(context: Context, expected_description: str) -> None:
|
||||
"""Verify the action description matches the expected value."""
|
||||
assert context.action_config is not None, "Action config should be loaded"
|
||||
assert context.action_config.description == expected_description, (
|
||||
f"Expected description '{expected_description}', got '{context.action_config.description}'"
|
||||
)
|
||||
|
||||
|
||||
@then('the action description should NOT contain "{text}"')
|
||||
def step_then_description_not_contains(context: Context, text: str) -> None:
|
||||
"""Verify the action description does not contain the specified text."""
|
||||
assert context.action_config is not None, "Action config should be loaded"
|
||||
assert text not in context.action_config.description, (
|
||||
f"Description should not contain '{text}', but got '{context.action_config.description}'"
|
||||
)
|
||||
|
||||
|
||||
@then('the loaded action strategy_actor should be "{expected_actor}"')
|
||||
def step_then_strategy_actor_is(context: Context, expected_actor: str) -> None:
|
||||
"""Verify the action strategy_actor matches the expected value."""
|
||||
assert context.action_config is not None, "Action config should be loaded"
|
||||
assert context.action_config.strategy_actor == expected_actor, (
|
||||
f"Expected strategy_actor '{expected_actor}', got '{context.action_config.strategy_actor}'"
|
||||
)
|
||||
|
||||
|
||||
@then('the action invariants should contain "{expected_invariant}"')
|
||||
def step_then_invariants_contain(context: Context, expected_invariant: str) -> None:
|
||||
"""Verify the action invariants contain the expected value."""
|
||||
assert context.action_config is not None, "Action config should be loaded"
|
||||
assert expected_invariant in context.action_config.invariants, (
|
||||
f"Expected invariant '{expected_invariant}' not found in {context.action_config.invariants}"
|
||||
)
|
||||
|
||||
|
||||
@then('the action invariants should NOT contain "{text}"')
|
||||
def step_then_invariants_not_contain(context: Context, text: str) -> None:
|
||||
"""Verify the action invariants do not contain the specified text."""
|
||||
assert context.action_config is not None, "Action config should be loaded"
|
||||
for invariant in context.action_config.invariants:
|
||||
assert text not in invariant, (
|
||||
f"Invariant should not contain '{text}', but got '{invariant}'"
|
||||
)
|
||||
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# Cleanup
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def cleanup_env_vars(context: Context) -> None:
|
||||
"""Clean up environment variables set during testing."""
|
||||
if hasattr(context, "env_vars_to_cleanup"):
|
||||
for var_name in context.env_vars_to_cleanup:
|
||||
if var_name in os.environ:
|
||||
del os.environ[var_name]
|
||||
context.env_vars_to_cleanup = []
|
||||
|
||||
|
||||
# Hook to cleanup after each scenario
|
||||
def after_scenario(context: Context, scenario: Any) -> None: # type: ignore[no-untyped-def]
|
||||
"""Clean up after each scenario."""
|
||||
cleanup_env_vars(context)
|
||||
@@ -477,4 +477,10 @@ def _interpolate_env_vars(data: dict[str, Any]) -> dict[str, Any]:
|
||||
def _env_replacer(match: re.Match[str]) -> str:
|
||||
"""Replace a single ``${VAR}`` match with its env value."""
|
||||
var_name = match.group(1)
|
||||
# Only allow CLEVERAGENTS_ prefixed variables with at least one char after prefix
|
||||
# (bare "CLEVERAGENTS_" with nothing after it is not a valid variable name)
|
||||
_prefix = "CLEVERAGENTS_"
|
||||
if not var_name.startswith(_prefix) or len(var_name) <= len(_prefix):
|
||||
return match.group(0)
|
||||
# Return the env value if it exists, otherwise leave as-is
|
||||
return os.environ.get(var_name, match.group(0))
|
||||
|
||||
@@ -38,7 +38,7 @@ if TYPE_CHECKING:
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_ENV_VAR = "CLEVERAGENTS_AUTOMATION_PROFILE"
|
||||
_DEFAULT_PROFILE = "manual"
|
||||
_DEFAULT_PROFILE = "supervised"
|
||||
|
||||
|
||||
class AutomationProfileService:
|
||||
@@ -62,7 +62,7 @@ class AutomationProfileService:
|
||||
When ``None``, only built-in profiles are available.
|
||||
global_default: Global default profile name from config.
|
||||
Falls back to ``CLEVERAGENTS_AUTOMATION_PROFILE``
|
||||
env var, then ``"manual"``.
|
||||
env var, then ``"supervised"``.
|
||||
"""
|
||||
self._repo = repo
|
||||
self._global_default = global_default
|
||||
@@ -71,7 +71,7 @@ class AutomationProfileService:
|
||||
def effective_global_default(self) -> str:
|
||||
"""Return the resolved global default profile name.
|
||||
|
||||
Precedence: explicit config > env var > ``"manual"``.
|
||||
Precedence: explicit config > env var > ``"supervised"``.
|
||||
"""
|
||||
if self._global_default:
|
||||
return self._global_default
|
||||
|
||||
Reference in New Issue
Block a user