Files
cleveractors-core/features/steps/credential_injection_steps.py
hurui200320 f281fa3b24
CI / lint (pull_request) Successful in 35s
CI / typecheck (pull_request) Successful in 50s
CI / security (pull_request) Successful in 50s
CI / quality (pull_request) Successful in 32s
CI / unit_tests (pull_request) Successful in 3m32s
CI / integration_tests (pull_request) Successful in 59s
CI / build (pull_request) Successful in 34s
CI / lint (push) Successful in 33s
CI / build (push) Successful in 37s
CI / quality (push) Successful in 48s
CI / typecheck (push) Successful in 48s
CI / security (push) Successful in 1m4s
CI / integration_tests (push) Successful in 1m18s
CI / coverage (pull_request) Successful in 3m34s
CI / status-check (pull_request) Successful in 3s
CI / unit_tests (push) Successful in 3m45s
CI / coverage (push) Successful in 3m34s
CI / status-check (push) Successful in 3s
feat(credentials): refactor LLMAgent/AgentFactory for per-request credential injection and extended provider routing
Implements Wave 3 of the cleveractors-core integration (ADR-2026, ADR-2028).

## AgentFactory
- Added optional credentials: dict[str, dict[str, str]] | None parameter to
  __init__. Stored as self.credentials.
- When creating LLMAgent instances, the full credentials dict is forwarded via
  a new credentials constructor kwarg, threading per-request API keys without
  touching the stored actor config_dict.
- Added explicit type annotation ReactiveAgentFactory: type[AgentFactory].
- _create_agent_instance raises ConfigurationError when credentials is not None
  but does not contain an entry for the requested provider (AC4 compliance).

## LLMAgent
- Added optional credentials: dict[str, str] | None parameter to __init__.
  Stored as self._credentials; config_dict is never modified.
- LangChain client construction deferred to first access (lazy init) via a
  chat_model property backed by _ensure_chat_model(). The property setter
  allows test-injection of mock models without going through the lazy-init path.
- _create_chat_model() dispatches to credential-injection / standalone paths.
- Extracted to llm_client.py and llm_providers.py for modularity.
- Defined shared module-level constants DEFAULT_MODEL, DEFAULT_TEMPERATURE,
  DEFAULT_MAX_TOKENS to eliminate duplicated magic numbers across llm.py,
  runtime.py, and factory.py.
- Added public credentials property exposing self._credentials.
- Added type narrowing guard in chat_model property to satisfy static analysis.
- Added temperature_override type validation in process_message.
- Guarded cleanup() null-assignment with _chat_model_lock (TOCTOU fix).

## SSRF Prevention (ADR-2028)
- _validate_base_url(): https-only, userinfo rejection, localhost rejection,
  raw IP literal rejection, numeric-hostname (dotted-hex/octal/compact) rejection,
  percent-decode loop with iteration cap (10), trailing-dot FQDN strip.
- DNS resolution intentionally omitted (blocking call in async path).
- Canonical URL reconstruction uses lowercased scheme.

## Credential Leak Prevention
- All logger.exception(...) replaced with logger.debug(..., exc_info=True).
- Error messages use type(e).__name__ instead of str(e).
- process_message exception chains broken with from None to prevent LangChain
  authentication errors from leaking via __cause__.

## Runtime fixes
- Executor._execute_llm: shallow copy for setdefault mutations.
- Executor._execute_graph: restructured try/finally for resource cleanup.
- Uses DEFAULT_MODEL, DEFAULT_TEMPERATURE, DEFAULT_MAX_TOKENS from llm.py.
- Defensive copy of credentials dict in Executor.__init__ to prevent caller
  mutation after construction.

## Test improvements
- Added BDD scenarios for llm_imports.py ImportError fallback branches using
  sys.modules manipulation with a custom import hook, exercised via subprocess.
- Added BDD scenario for AgentFactory.create_agents_from_config() credential
  threading to all created agents.
- Removed tautological dataclass test scenarios (NodeUsage, ActorResult) from
  runtime_coverage.feature.
- Fixed misleading scenario title in runtime_coverage.feature (exception chain
  suppressed, not chained) and updated step to assert __cause__ is None.
- Added CHANGELOG entry for ReactiveAgentFactory backward-compatibility alias.
- Removed duplicate log emission in _check_known_provider_domain.

ISSUES CLOSED: #12
2026-06-08 11:10:46 +00:00

455 lines
17 KiB
Python

"""Step definitions for per-request credential injection (core factory + lazy init).
Covers AgentFactory credentials parameter, LLMAgent lazy init, agent cache
behaviour, factory-to-agent credential threading, process_message end-to-end,
and cleanup.
Provider routing scenarios live in credential_native_provider_steps.py and
credential_ssrf_steps.py. Temperature override and __repr__ redaction
scenarios live in credential_agent_state_steps.py.
Executor-level tests live in credential_executor_steps.py.
"""
from __future__ import annotations
from typing import Any, cast
from unittest.mock import MagicMock, patch
from behave import given, step, then, when
from behave.api.async_step import async_run_until_complete
from features.mocks.credential_helpers import (
make_template_renderer,
minimal_actor_config,
mock_chat_model,
)
from cleveractors.agents.factory import AgentFactory
from cleveractors.agents.llm import LLMAgent
from cleveractors.core.exceptions import (
AgentCreationError,
ConfigurationError,
ExecutionError,
)
# ---------------------------------------------------------------------------
# Shared Given step — used by factory, composite, and validation scenarios
# ---------------------------------------------------------------------------
@given("a minimal actor config with a single openai LLM agent")
def step_minimal_actor_config(context: Any) -> None:
"""Set up a minimal actor config."""
context.actor_config = minimal_actor_config()
context.template_renderer = make_template_renderer()
# ---------------------------------------------------------------------------
# LLMAgent lazy init scenarios
# ---------------------------------------------------------------------------
@given(
"I construct an LLMAgent for openai with api_key in config and no credentials dict"
)
def step_llm_agent_openai_standalone(context: Any) -> None:
"""Construct an LLMAgent in standalone mode (no credentials dict)."""
context.agent_config = {
"provider": "openai",
"api_key": "standalone-api-key",
}
context.renderer = make_template_renderer()
context.agent = LLMAgent(
name="test_agent",
config=context.agent_config,
template_renderer=context.renderer,
)
@then("the internal _chat_model attribute should be None")
def step_chat_model_none(context: Any) -> None:
"""Verify _chat_model is None (lazy init not yet triggered)."""
assert context.agent._chat_model is None, (
f"Expected _chat_model to be None immediately after construction, "
f"got {context.agent._chat_model!r}"
)
@when("I access the chat_model property")
def step_access_chat_model_property(context: Any) -> None:
"""Access the chat_model property (triggers lazy init).
For the google credential-injection test, we patch
ChatGoogleGenerativeAI at access time. For non-native provider
routing tests using test domains, we bypass _validate_base_url
DNS resolution since those tests are about routing, not SSRF
prevention (which has its own dedicated scenarios).
"""
patches: list[Any] = []
if getattr(context, "_need_google_patch", False):
mock_google_cls = MagicMock()
mock_google_cls.return_value = MagicMock()
patches.append(
patch("cleveractors.agents.llm.ChatGoogleGenerativeAI", mock_google_cls)
)
if getattr(context, "_skip_base_url_validation", False):
from urllib.parse import urlparse
from cleveractors.agents import llm_client as lc_module
patches.append(
patch.object(
lc_module,
"_validate_base_url",
side_effect=lambda url: (url, urlparse(url).hostname),
)
)
# Activate all patches, access property, then clean up
for p in patches:
p.start()
try:
context.accessed_chat_model = context.agent.chat_model
finally:
for p in reversed(patches):
p.stop()
@then("the internal _chat_model attribute should not be None")
def step_chat_model_not_none(context: Any) -> None:
"""Verify _chat_model is set after property access."""
assert context.agent._chat_model is not None, (
"Expected _chat_model to be set after property access, but it is still None"
)
@when("I assign a mock object to the chat_model property")
def step_assign_mock_to_chat_model(context: Any) -> None:
"""Assign a mock to the chat_model property (setter path)."""
context.injected_mock = MagicMock()
context.agent.chat_model = context.injected_mock
@then("the _chat_model attribute should be the mock object")
def step_chat_model_is_mock(context: Any) -> None:
"""Verify _chat_model is the injected mock."""
assert context.agent._chat_model is context.injected_mock, (
"Expected _chat_model to be the injected mock"
)
@then("accessing chat_model again returns the same mock")
def step_chat_model_getter_returns_mock(context: Any) -> None:
"""Verify getter returns injected mock without triggering lazy init."""
result = context.agent.chat_model
assert result is context.injected_mock, (
"Expected chat_model getter to return the injected mock"
)
# ---------------------------------------------------------------------------
# AgentFactory threads credentials to LLMAgent
# ---------------------------------------------------------------------------
@step("I call create_agent for the llm agent")
def step_factory_create_agent(context: Any) -> None:
"""Call factory.create_agent for the LLM agent name."""
context.created_agent = context.factory.create_agent("my_agent")
@then("the created LLMAgent should have the credentials dict set")
def step_created_agent_has_credentials(context: Any) -> None:
"""Verify the created LLMAgent has the provider-specific credentials slice."""
assert isinstance(context.created_agent, LLMAgent), (
f"Expected LLMAgent, got {type(context.created_agent).__name__}"
)
# The factory extracts the provider-specific slice from the credentials dict
# (e.g. credentials["openai"] → {"api_key": "injected-key"}).
expected = context.credentials.get("openai")
assert context.created_agent._credentials == expected, (
f"Expected _credentials {expected!r}, "
f"got {context.created_agent._credentials!r}"
)
@when("I call create_agent for the llm agent with api_key in config")
def step_factory_create_agent_with_api_key(context: Any) -> None:
"""Call factory.create_agent; agent has api_key in config (standalone mode)."""
context.actor_config["agents"]["my_agent"]["config"]["api_key"] = "test-key"
context.factory = AgentFactory(
config=context.actor_config,
template_renderer=context.template_renderer,
)
context.created_agent = context.factory.create_agent("my_agent")
@then("the created LLMAgent should have credentials equal to None")
def step_created_agent_credentials_none(context: Any) -> None:
"""Verify the created LLMAgent has credentials=None when factory had none."""
assert isinstance(context.created_agent, LLMAgent), (
f"Expected LLMAgent, got {type(context.created_agent).__name__}"
)
assert context.created_agent._credentials is None, (
f"Expected _credentials to be None, "
f"got {type(context.created_agent._credentials).__name__}"
)
# ---------------------------------------------------------------------------
# process_message end-to-end with credentials
# ---------------------------------------------------------------------------
@given('I inject a mock chat model that returns "{response_text}"')
def step_inject_mock_chat_model(context: Any, response_text: str) -> None:
"""Inject a mock chat model that returns the specified response."""
context.expected_response = response_text
context.agent.chat_model = mock_chat_model(response_text)
@when('I call process_message with "{message}"')
@async_run_until_complete
async def step_call_process_message(context: Any, message: str) -> None:
"""Call process_message and store the result."""
context.process_result = await context.agent.process_message(message)
@when('I call process_message with "{message}" (expecting error)')
@async_run_until_complete
async def step_call_process_message_expecting_error(context: Any, message: str) -> None:
"""Call process_message and capture any exception raised."""
try:
context.process_result = await context.agent.process_message(message)
context.raised_exception = None
except ConfigurationError as exc:
context.raised_exception = exc
except Exception as exc:
context.raised_exception = exc
@then('the response should be "{expected_text}"')
def step_response_should_be(context: Any, expected_text: str) -> None:
"""Verify the process_message result matches the expected text."""
assert context.process_result == expected_text, (
f"Expected response {expected_text!r}, got {context.process_result!r}"
)
# ---------------------------------------------------------------------------
# cleanup() when _chat_model is None
# ---------------------------------------------------------------------------
@when("I call cleanup on the agent")
@async_run_until_complete
async def step_call_cleanup(context: Any) -> None:
"""Call cleanup on the agent."""
context.cleanup_error = None
try:
await context.agent.cleanup()
except Exception as e:
context.cleanup_error = e
@when("I call cleanup on the agent again")
@async_run_until_complete
async def step_call_cleanup_again(context: Any) -> None:
"""Call cleanup on the agent a second time (idempotency test)."""
try:
await context.agent.cleanup()
except Exception as e:
context.cleanup_error = e
@then("no error should occur and _chat_model should remain None")
def step_cleanup_noop(context: Any) -> None:
"""Verify cleanup did not raise and _chat_model is still None."""
assert context.cleanup_error is None, (
f"Expected no error from cleanup, got {context.cleanup_error!r}"
)
assert context.agent._chat_model is None, (
f"Expected _chat_model to remain None after cleanup, "
f"got {context.agent._chat_model!r}"
)
@given(
"I construct an LLMAgent for openai with credentials dict containing only "
"api_key and config containing temperature 0.0"
)
def step_openai_agent_with_creds_and_temperature_zero(context: Any) -> None:
"""Construct LLMAgent with temperature=0.0 — a valid, falsy value."""
context.credentials = {"api_key": "injected-openai-key"}
config: dict[str, Any] = {
"provider": "openai",
"model": "gpt-3.5-turbo",
"temperature": 0.0,
}
context.renderer = make_template_renderer()
context.agent = LLMAgent(
name="test_openai_temp_zero",
config=config,
template_renderer=context.renderer,
credentials=context.credentials,
)
@then("the agent temperature should be 0.0")
def step_agent_temperature_is_zero(context: Any) -> None:
"""Verify temperature is exactly 0.0 — not replaced by default 0.7."""
assert context.agent.temperature == 0.0, (
f"Expected temperature 0.0, got {context.agent.temperature}"
)
# ---------------------------------------------------------------------------
# M2: validate_credentials_structure error paths (AgentFactory)
# ---------------------------------------------------------------------------
@given(
'I pass a credentials dict with a non-dict value for "{provider}" to AgentFactory'
)
def step_factory_non_dict_cred_entry(context: Any, provider: str) -> None:
"""Prepare credentials where an entry is not a dict.
The actual AgentFactory construction (and error capture) happens in the
When step — this Given step only prepares data.
"""
context.actor_config = minimal_actor_config()
context.template_renderer = make_template_renderer()
context.credentials = {
provider: cast(dict[str, str], "not-a-dict")
} # intentionally invalid
@given(
"I pass a credentials dict with a non-str inner value for "
'"{provider}".api_key to AgentFactory'
)
def step_factory_non_str_inner_value(context: Any, provider: str) -> None:
"""Prepare credentials where inner api_key is not a str.
The actual AgentFactory construction (and error capture) happens in the
When step — this Given step only prepares data.
"""
context.actor_config = minimal_actor_config()
context.template_renderer = make_template_renderer()
context.credentials = {
provider: cast(dict[str, str], {"api_key": 12345})
} # intentionally invalid
@when("I call create_agent for the llm agent (expecting error)")
def step_factory_create_agent_expecting_error(context: Any) -> None:
"""Call factory.create_agent and capture any exception."""
try:
context.created_agent = context.factory.create_agent("my_agent")
context.raised_exception = None
except ConfigurationError as exc:
context.raised_exception = exc
except Exception as exc:
context.raised_exception = exc
@when("I create an AgentFactory with the prepared credentials (expecting error)")
def step_factory_create_with_bad_credentials(context: Any) -> None:
"""Create AgentFactory with prepared bad credentials; capture any error."""
try:
context.factory = AgentFactory(
config=context.actor_config,
template_renderer=context.template_renderer,
credentials=context.credentials,
)
context.raised_exception = None
except ConfigurationError as exc:
context.raised_exception = exc
context.factory = None
# ---------------------------------------------------------------------------
# m4: AgentCreationError wrapping in factory._create_agent_instance
# ---------------------------------------------------------------------------
@given("I patch LLMAgent.__init__ to raise RuntimeError")
def step_patch_llm_agent_init_runtime_error(context: Any) -> None:
"""Patch LLMAgent.__init__ to raise RuntimeError so we can test
that AgentFactory wraps unexpected exceptions in AgentCreationError.
"""
patcher = patch.object(
LLMAgent,
"__init__",
side_effect=RuntimeError("Unexpected constructor failure"),
)
patcher.start()
if not hasattr(context, "_active_patches"):
context._active_patches = []
context._active_patches.append(patcher)
# ---------------------------------------------------------------------------
# Generic Then step for non-ConfigurationError exception assertions
# ---------------------------------------------------------------------------
@then('the raised error should be an AgentCreationError containing "{expected_text}"')
def step_agent_creation_error_contains(context: Any, expected_text: str) -> None:
"""Verify the captured exception is AgentCreationError with expected message."""
assert context.raised_exception is not None, (
"Expected an error to be raised, but none was captured"
)
assert isinstance(context.raised_exception, AgentCreationError), (
f"Expected AgentCreationError, got {type(context.raised_exception).__name__}: "
f"{context.raised_exception!r}"
)
assert expected_text in str(context.raised_exception), (
f"Expected error message to contain {expected_text!r}, "
f"got {str(context.raised_exception)!r}"
)
@then('the raised error should be an ExecutionError containing "{expected_text}"')
def step_execution_error_contains(context: Any, expected_text: str) -> None:
"""Verify the captured exception is ExecutionError with expected message."""
assert context.raised_exception is not None, (
"Expected an error to be raised, but none was captured"
)
assert isinstance(context.raised_exception, ExecutionError), (
f"Expected ExecutionError, got {type(context.raised_exception).__name__}: "
f"{context.raised_exception!r}"
)
assert expected_text in str(context.raised_exception), (
f"Expected error message to contain {expected_text!r}, "
f"got {str(context.raised_exception)!r}"
)
# ---------------------------------------------------------------------------
# M2: Inner-dict mutation protection
# ---------------------------------------------------------------------------
@given('I mutate the original credentials dict\'s api_key to "{mutated_value}"')
def step_mutate_credentials_api_key(context: Any, mutated_value: str) -> None:
"""Mutate the original (caller-side) credentials dict api_key value.
The stored credentials should be unaffected because
LLMAgent.__init__ shallow-copies the credentials dict.
"""
context.credentials["api_key"] = mutated_value
@then("the agent's stored api_key should still be the original value")
def step_stored_api_key_unchanged(context: Any) -> None:
"""Verify the agent's stored api_key was not affected by caller mutation."""
creds = context.agent._credentials
assert creds is not None, "Expected credentials to be set on agent"
actual = creds.get("api_key")
assert actual == "injected-openai-key", (
f"Expected stored api_key to be 'injected-openai-key' (original value), "
f"got {actual!r}"
)