hurui200320
ed9c4dc95d
CI / quality (pull_request) Successful in 34s
CI / build (pull_request) Successful in 57s
CI / integration_tests (pull_request) Successful in 1m26s
CI / unit_tests (pull_request) Successful in 3m28s
CI / lint (pull_request) Successful in 1m1s
CI / typecheck (pull_request) Successful in 1m0s
CI / security (pull_request) Successful in 59s
CI / coverage (pull_request) Successful in 3m7s
CI / status-check (pull_request) Successful in 3s
CI / lint (push) Successful in 40s
CI / quality (push) Successful in 40s
CI / build (push) Successful in 40s
CI / typecheck (push) Successful in 1m5s
CI / security (push) Successful in 1m5s
CI / integration_tests (push) Successful in 1m20s
CI / unit_tests (push) Successful in 3m14s
CI / coverage (push) Successful in 3m2s
CI / status-check (push) Successful in 3s
LLMAgent.cleanup() previously iterated over hard-coded provider SDK client attributes (root_async_client, root_client, _async_client, _client) and called close() on each. Recent langchain-anthropic and langchain-openai versions cache their default httpx clients via module-level lru_cache functions. Closing those clients poisoned the cache: every subsequent ChatAnthropic/ChatOpenAI instance in the same process received the same closed httpx client and failed with a connection error. Fix: cleanup() now only releases the agent's own reference to the chat model (self._chat_model = None). The removed _KNOWN_CLIENT_ATTRS class variable has been deleted and ClassVar removed from the typing import. The concurrent-idempotency guarantee is preserved: the lock is still acquired before nulling _chat_model, so two concurrent cleanup() calls cannot both see a non-None model and attempt conflicting operations. The four provider SDK client-closing scenarios in credential_injection.feature and llm_missing_coverage.feature are removed as they tested the old (buggy) behaviour. Their step definitions are removed from credential_cleanup_steps.py (now only carries the resolve_class_ref patch step) and llm_missing_coverage_steps.py is updated with the corrected assertions. Six new regression BDD scenarios tagged @tdd_issue @tdd_issue_57 are added in features/llm_cleanup_shared_client.feature, covering all four provider SDK client attribute paths (Anthropic _async_client/_client, OpenAI root_async_client/root_client) and two end-to-end two-agent scenarios that prove a second agent can run successfully after the first is cleaned up. ISSUES CLOSED: #57
912 lines
52 KiB
Gherkin
912 lines
52 KiB
Gherkin
Feature: Per-Request Credential Injection and Extended Provider Routing
|
||
As a CleverThis router developer
|
||
I want LLMAgent and AgentFactory to accept a credentials dict for per-request injection
|
||
So that API keys can be supplied at call time without environment variables or config mutation
|
||
|
||
# Acceptance criteria from issue #12 / ADR-2026 / ADR-2028
|
||
|
||
# ------------------------------------------------------------------
|
||
# AgentFactory credentials parameter
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: AgentFactory accepts credentials parameter
|
||
Given a minimal actor config with a single openai LLM agent
|
||
When I create an AgentFactory with a credentials dict for openai
|
||
Then the factory should store the credentials dict
|
||
|
||
Scenario: AgentFactory stores None when credentials not supplied
|
||
Given a minimal actor config with a single openai LLM agent
|
||
When I create an AgentFactory without credentials
|
||
Then the factory credentials should be None
|
||
|
||
# ------------------------------------------------------------------
|
||
# LLMAgent lazy init (chat_model property)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: LLMAgent chat_model is None immediately after construction
|
||
Given I construct an LLMAgent for openai with api_key in config and no credentials dict
|
||
Then the internal _chat_model attribute should be None
|
||
|
||
Scenario: LLMAgent chat_model property triggers lazy init on first access
|
||
Given I construct an LLMAgent for openai with api_key in config and no credentials dict
|
||
When I access the chat_model property
|
||
Then the internal _chat_model attribute should not be None
|
||
|
||
Scenario: LLMAgent chat_model setter bypasses lazy init
|
||
Given I construct an LLMAgent for openai with api_key in config and no credentials dict
|
||
When I assign a mock object to the chat_model property
|
||
Then the _chat_model attribute should be the mock object
|
||
And accessing chat_model again returns the same mock
|
||
|
||
# ------------------------------------------------------------------
|
||
# Credential-injection path — native providers
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: LLMAgent uses injected credentials for openai instead of env var
|
||
Given I construct an LLMAgent for openai with credentials dict containing only api_key
|
||
When I access the chat_model property
|
||
When I rebuild the chat model while spying on os.getenv
|
||
Then no environment variables should have been consulted for the API key
|
||
|
||
Scenario: LLMAgent uses injected credentials for anthropic instead of env var
|
||
Given I construct an LLMAgent for anthropic with credentials dict containing only api_key
|
||
When I access the chat_model property
|
||
When I rebuild the chat model while spying on os.getenv
|
||
Then no environment variables should have been consulted for the API key
|
||
|
||
Scenario: LLMAgent uses injected credentials for google instead of env var
|
||
Given I construct an LLMAgent for google with credentials dict containing api_key
|
||
When I access the chat_model property
|
||
When I rebuild the chat model while spying on os.getenv
|
||
Then no environment variables should have been consulted for the API key
|
||
|
||
# ------------------------------------------------------------------
|
||
# Credential-injection path — extended provider routing (ADR-2028)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Non-native provider with base_url routes to ChatOpenAI
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url
|
||
When I access the chat_model property
|
||
Then the created model should be a ChatOpenAI instance
|
||
And the model should have the correct base_url from credentials
|
||
And the model should have the correct model_name, temperature, and max_tokens from the agent config
|
||
|
||
Scenario: openai_compatible provider routes to ChatOpenAI with base_url
|
||
Given I construct an LLMAgent for provider "openai_compatible" with credentials containing api_key and base_url
|
||
When I access the chat_model property
|
||
Then the created model should be a ChatOpenAI instance
|
||
And the model should have the correct base_url from credentials
|
||
And the model should have the correct model_name, temperature, and max_tokens from the agent config
|
||
|
||
Scenario: Named non-native provider "fireworks" routes to ChatOpenAI
|
||
Given I construct an LLMAgent for provider "fireworks" with credentials containing api_key and base_url
|
||
When I access the chat_model property
|
||
Then the created model should be a ChatOpenAI instance
|
||
|
||
Scenario: Named non-native provider "together" routes to ChatOpenAI
|
||
Given I construct an LLMAgent for provider "together" with credentials containing api_key and base_url
|
||
When I access the chat_model property
|
||
Then the created model should be a ChatOpenAI instance
|
||
|
||
# ------------------------------------------------------------------
|
||
# Error: missing credentials for provider
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: ConfigurationError when credentials dict supplied but provider absent
|
||
Given I construct an LLMAgent for provider "groq" with an empty credentials dict
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "missing 'api_key'"
|
||
|
||
Scenario: ConfigurationError when credentials dict has different provider key
|
||
Given I construct an LLMAgent for provider "anthropic" with credentials only for "openai"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "missing 'api_key'"
|
||
|
||
Scenario: ConfigurationError for non-native provider without base_url in credentials
|
||
Given I construct an LLMAgent for provider "groq" with credentials missing base_url
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "missing 'base_url'"
|
||
|
||
# ------------------------------------------------------------------
|
||
# Base URL validation (ADR-2028 SSRF prevention)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: base_url validation rejects non-https scheme
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "http://api.example.com/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "scheme must be 'https'"
|
||
|
||
Scenario: base_url validation rejects userinfo in URL
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://user:pass@api.example.com/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not contain userinfo"
|
||
|
||
Scenario: base_url validation rejects raw IP address
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://127.0.0.1:8080/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use a raw IP address"
|
||
|
||
# ------------------------------------------------------------------
|
||
# M3: SSRF Bypass — localhost and hex/decimal IPv4 representations
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: base_url validation rejects localhost
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://localhost:8080/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use 'localhost'"
|
||
|
||
Scenario: base_url validation rejects localhost subdomain
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://api.localhost/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use 'localhost'"
|
||
|
||
Scenario: base_url validation rejects hex-encoded IPv4 address
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://0x7f000001/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use a numeric IP representation"
|
||
|
||
Scenario: base_url validation rejects decimal IPv4 address
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://2130706433/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use a numeric IP representation"
|
||
|
||
Scenario: base_url validation rejects URL with no hostname
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https:///v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must include a valid hostname"
|
||
|
||
# ------------------------------------------------------------------
|
||
# C1: SSRF Bypass — dotted-hex/octal/compact-decimal IPv4
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: base_url validation rejects dotted-hex IPv4 address
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://0x7f.0.0.1/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use a numeric IP representation"
|
||
|
||
Scenario: base_url validation rejects dotted-octal IPv4 address
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://0177.0.0.1/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use a numeric IP representation"
|
||
|
||
Scenario: base_url validation rejects compact-decimal IPv4 address
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://127.1/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use a numeric IP representation"
|
||
|
||
# ------------------------------------------------------------------
|
||
# cleanup() idempotency when chat_model was accessed (m4)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: cleanup is idempotent when called twice after chat_model was accessed
|
||
Given I construct an LLMAgent for openai with credentials dict containing only api_key
|
||
And I inject a mock chat model that returns "test"
|
||
When I call cleanup on the agent
|
||
And I call cleanup on the agent again
|
||
Then no error should occur and _chat_model should remain None
|
||
|
||
# ------------------------------------------------------------------
|
||
# m3: IPv6 loopback SSRF validation
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: base_url validation rejects IPv6 loopback
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://[::1]/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use a raw IP address"
|
||
|
||
# ------------------------------------------------------------------
|
||
# Env-var fallback still works (standalone / CLI mode)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Standalone mode uses OPENAI_API_KEY env var when no credentials dict
|
||
Given the OPENAI_API_KEY environment variable is set to "sk-env-test"
|
||
And I construct an LLMAgent for openai without credentials dict and without api_key in config
|
||
When I access the chat_model property
|
||
Then the chat_model should be created successfully
|
||
|
||
Scenario: Standalone mode raises ConfigurationError when env var missing and no api_key
|
||
Given all API key environment variables are cleared
|
||
And I construct an LLMAgent for openai without credentials dict and without api_key in config
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "Missing API key"
|
||
|
||
Scenario: Standalone mode raises ConfigurationError for non-native provider
|
||
Given I construct an LLMAgent for provider "groq" without credentials dict and with api_key in config
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "Unsupported provider"
|
||
|
||
Scenario: Standalone mode rejects whitespace-only api_key in config
|
||
Given I construct an LLMAgent for openai without credentials dict and with whitespace-only api_key in config
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "Missing API key"
|
||
|
||
# ------------------------------------------------------------------
|
||
# config_dict never modified
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: config_dict is not modified by credential injection
|
||
Given I construct an LLMAgent for openai with credentials dict containing only api_key
|
||
And I note the original config_dict state
|
||
When I access the chat_model property
|
||
Then the config_dict should equal the noted original state
|
||
|
||
# ------------------------------------------------------------------
|
||
# AgentFactory threads credentials to LLMAgent
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: AgentFactory passes credentials to LLMAgent during create_agent
|
||
Given a minimal actor config with a single openai LLM agent
|
||
And I create an AgentFactory with a credentials dict for openai
|
||
When I call create_agent for the llm agent
|
||
Then the created LLMAgent should have the credentials dict set
|
||
|
||
Scenario: AgentFactory without credentials creates LLMAgent with None credentials
|
||
Given a minimal actor config with a single openai LLM agent
|
||
When I create an AgentFactory without credentials
|
||
And I call create_agent for the llm agent with api_key in config
|
||
Then the created LLMAgent should have credentials equal to None
|
||
|
||
Scenario: AgentFactory wraps unexpected exceptions in AgentCreationError
|
||
Given a minimal actor config with a single openai LLM agent
|
||
And I create an AgentFactory without credentials
|
||
And I patch LLMAgent.__init__ to raise RuntimeError
|
||
When I call create_agent for the llm agent (expecting error)
|
||
Then the raised error should be an AgentCreationError containing "Failed to create agent"
|
||
|
||
# ------------------------------------------------------------------
|
||
# process_message works end-to-end with injected credentials
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: process_message succeeds with injected openai credentials and mock chat model
|
||
Given I construct an LLMAgent for openai with credentials dict containing only api_key
|
||
And I inject a mock chat model that returns "Injected credential response"
|
||
When I call process_message with "Hello from credential injection"
|
||
Then the response should be "Injected credential response"
|
||
|
||
Scenario: process_message succeeds with non-native provider credentials and mock
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url
|
||
And I inject a mock chat model that returns "Groq response via base_url"
|
||
When I call process_message with "Hello from groq"
|
||
Then the response should be "Groq response via base_url"
|
||
|
||
# ------------------------------------------------------------------
|
||
# ADR-2028 Tier 1 providers (M-3)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Named non-native provider "openrouter" routes to ChatOpenAI
|
||
Given I construct an LLMAgent for provider "openrouter" with credentials containing api_key and base_url
|
||
When I access the chat_model property
|
||
Then the created model should be a ChatOpenAI instance
|
||
|
||
Scenario: Named non-native provider "mistral" routes to ChatOpenAI
|
||
Given I construct an LLMAgent for provider "mistral" with credentials containing api_key and base_url
|
||
When I access the chat_model property
|
||
Then the created model should be a ChatOpenAI instance
|
||
|
||
# ------------------------------------------------------------------
|
||
# cleanup() when _chat_model is None (M-4)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: cleanup is a no-op when chat_model was never accessed
|
||
Given I construct an LLMAgent for openai with credentials dict containing only api_key
|
||
When I call cleanup on the agent
|
||
Then no error should occur and _chat_model should remain None
|
||
|
||
# ------------------------------------------------------------------
|
||
# process_message raises ConfigurationError from lazy init (M-5)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: process_message raises ConfigurationError when credentials are missing
|
||
Given I construct an LLMAgent for provider "groq" with an empty credentials dict
|
||
When I call process_message with "Hello" (expecting error)
|
||
Then a credential ConfigurationError should contain "missing 'api_key'"
|
||
|
||
# ------------------------------------------------------------------
|
||
# Native provider ignores base_url in credentials (M-6)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Native openai provider ignores base_url in credentials dict
|
||
Given I construct an LLMAgent for openai with credentials containing api_key and base_url
|
||
And I install a log-capture handler on the llm_client logger for native provider warning
|
||
When I access the chat_model property
|
||
Then the created model should be a ChatOpenAI instance
|
||
And the model should NOT have the base_url from credentials
|
||
And a warning log should be emitted about base_url being ignored for native provider
|
||
|
||
# ------------------------------------------------------------------
|
||
# ConfigurationError for native provider with empty api_key (M-2)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: ConfigurationError for native provider with empty api_key in credentials
|
||
Given I construct an LLMAgent for openai with credentials containing empty api_key
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "missing 'api_key'"
|
||
|
||
Scenario: ConfigurationError for native provider with missing api_key key in credentials
|
||
Given I construct an LLMAgent for openai with credentials containing no api_key key
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "missing 'api_key'"
|
||
|
||
# ------------------------------------------------------------------
|
||
# AgentFactory skips cache when credentials are set (M-10)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: AgentFactory does not cache agents when credentials are not None
|
||
Given a minimal actor config with a single openai LLM agent
|
||
And I create an AgentFactory with a credentials dict for openai
|
||
When I call create_agent for the llm agent
|
||
And I call create_agent for the llm agent again
|
||
Then two distinct LLMAgent instances should have been created
|
||
|
||
# ------------------------------------------------------------------
|
||
# Executor: credential injection via AgentFactory (ADR-2026, AC8)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Executor._execute_llm routes through AgentFactory with credentials
|
||
Given an Executor for an LLM actor with openai credentials dict
|
||
When I execute the LLM actor with "Hello executor"
|
||
Then the LLM execution should succeed with the mock response
|
||
And the executor LLM actor config_dict should be unchanged after execution
|
||
|
||
Scenario: Executor.execute propagates ConfigurationError for missing LLM credentials
|
||
Given an Executor for an LLM actor with an empty credentials dict
|
||
When I execute the LLM actor expecting ConfigurationError
|
||
Then a credential ConfigurationError should contain "missing credentials for provider"
|
||
|
||
Scenario: Executor wraps unexpected agent errors in ExecutionError
|
||
Given an Executor for an LLM actor with openai credentials dict
|
||
And I patch the agent process_message to raise RuntimeError
|
||
When I execute the LLM actor for ExecutionError wrapping test
|
||
Then the raised error should be an ExecutionError containing "LLM execution failed"
|
||
|
||
Scenario: Executor._execute_graph routes through AgentFactory with credentials
|
||
Given an Executor for a graph actor with openai credentials dict
|
||
When I execute the graph actor with "Hello from graph"
|
||
Then the graph execution should succeed with the mock graph response
|
||
And the AgentFactory should have received the graph credentials dict
|
||
|
||
Scenario: Executor._execute_graph does not inject credentials into config_dict (AC8)
|
||
Given an Executor for a graph actor with openai credentials dict
|
||
And I record the graph actor config_dict state
|
||
When I execute the graph actor with "Hello from graph"
|
||
Then the executor graph actor config_dict should be unchanged after execution
|
||
|
||
# ------------------------------------------------------------------
|
||
# SSRF Bypass: Percent-Encoded IP Literals (ADR-2028, C1)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: base_url validation decodes percent-encoded IP addresses
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://%31%32%37%2e%30%2e%30%2e%31/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use a raw IP address"
|
||
|
||
Scenario: base_url validation decodes double-percent-encoded IP addresses
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://%2531%2532%2537%252e%2530%252e%2530%252e%2531/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use a raw IP address"
|
||
|
||
Scenario: base_url validation rejects trailing-dot IP address
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://127.0.0.1./v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use a raw IP address"
|
||
|
||
# ------------------------------------------------------------------
|
||
# Known provider domain pattern checking (ADR-2028 §Design, M2)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Known provider domain pattern matches expected pattern
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://api.groq.com/v1"
|
||
When I access the chat_model property
|
||
Then the created model should be a ChatOpenAI instance
|
||
And no domain-pattern warning should have been emitted
|
||
|
||
Scenario: Known provider domain pattern warns on unexpected hostname
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://unexpected-host.evil.com/v1"
|
||
When I access the chat_model property
|
||
Then a warning log should be emitted about unexpected domain pattern
|
||
|
||
Scenario: Unknown provider domain pattern triggers catalog-not-found warning
|
||
Given I construct an LLMAgent for provider "custom_provider" with credentials containing api_key and base_url "https://api.custom-provider.example.com/v1"
|
||
When I access the chat_model property
|
||
Then the created model should be a ChatOpenAI instance
|
||
And a warning log should be emitted about unknown provider domain
|
||
|
||
# ------------------------------------------------------------------
|
||
# _resolve_class_ref error path (m7)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: _resolve_class_ref raises ConfigurationError for unknown class name
|
||
Given I construct an LLMAgent for openai with api_key in config and no credentials dict
|
||
And I patch _resolve_class_ref to raise ConfigurationError for an invalid class name
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "Unknown class reference"
|
||
|
||
# ------------------------------------------------------------------
|
||
# Composite agent credential threading (m8)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: AgentFactory passes credentials to nested LLMAgent in composite agent
|
||
Given a composite agent config with a nested LLM component for credential test
|
||
When I call create_agent for the composite agent in credential test
|
||
Then the nested LLM agent within the composite should have the credentials dict set
|
||
|
||
# ------------------------------------------------------------------
|
||
# m2: Legacy composite agent path with credentials
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: AgentFactory passes credentials to nested LLMAgent via legacy composite path
|
||
Given a legacy composite agent config with nested agents list for credential test
|
||
When I call create_agent for the legacy composite agent in credential test
|
||
Then the nested LLM agent within the legacy composite should have the credentials dict set
|
||
|
||
# ------------------------------------------------------------------
|
||
# n4: temperature=0.0 is a valid, falsy value
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: LLMAgent constructor respects temperature=0.0
|
||
Given I construct an LLMAgent for openai with credentials dict containing only api_key and config containing temperature 0.0
|
||
Then the agent temperature should be 0.0
|
||
|
||
# ------------------------------------------------------------------
|
||
# M2: validate_credentials_structure error paths
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: LLMAgent constructor rejects non-dict credentials
|
||
Given I pass a non-dict credentials value "a string" to LLMAgent
|
||
When I construct the LLMAgent with the prepared non-dict credentials (expecting error)
|
||
Then a credential ConfigurationError should contain "credentials must be a dict"
|
||
|
||
Scenario: AgentFactory constructor rejects non-dict credential entry
|
||
Given I pass a credentials dict with a non-dict value for "openai" to AgentFactory
|
||
When I create an AgentFactory with the prepared credentials (expecting error)
|
||
Then a credential ConfigurationError should contain "must be a dict"
|
||
|
||
Scenario: AgentFactory constructor rejects non-str credential inner value
|
||
Given I pass a credentials dict with a non-str inner value for "openai".api_key to AgentFactory
|
||
When I create an AgentFactory with the prepared credentials (expecting error)
|
||
Then a credential ConfigurationError should contain "must be a str"
|
||
|
||
# ------------------------------------------------------------------
|
||
# M3: whitespace-only api_key
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: ConfigurationError for native provider with whitespace-only api_key in credentials
|
||
Given I construct an LLMAgent for openai with credentials containing whitespace-only api_key
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "missing 'api_key'"
|
||
|
||
# ------------------------------------------------------------------
|
||
# m6: legacy composite path without credentials
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: AgentFactory creates legacy composite without credentials dict
|
||
Given a legacy composite agent config with nested agents list without credentials
|
||
When I call create_agent for the legacy composite agent in credential test
|
||
Then the nested LLM agent within the legacy composite should have credentials equal to None
|
||
|
||
# ------------------------------------------------------------------
|
||
# M1: Null-byte SSRF bypass via percent-decoded control characters
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: base_url validation rejects null-byte appended to loopback IP
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://127.0.0.1%00/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "invalid control characters"
|
||
|
||
Scenario: base_url validation rejects null-byte appended to localhost
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://localhost%00/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "invalid control characters"
|
||
|
||
# ------------------------------------------------------------------
|
||
# n3: IPv4-mapped IPv6 loopback
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: base_url validation rejects IPv4-mapped IPv6 loopback
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://[::ffff:127.0.0.1]/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use a raw IP address"
|
||
|
||
# ------------------------------------------------------------------
|
||
# n4: Percent-encoded localhost
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: base_url validation rejects percent-encoded localhost
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://%6c%6f%63%61%6c%68%6f%73%74/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use 'localhost'"
|
||
|
||
# ------------------------------------------------------------------
|
||
# C1: SSRF Bypass — percent-encoded bracket-wrapped IPv6 literals
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: base_url validation decodes percent-encoded bracket-wrapped IPv6 loopback
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://%5b%3a%3a1%5d/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use a raw IP address"
|
||
|
||
Scenario: base_url validation decodes percent-encoded bracket-wrapped IPv4-mapped IPv6
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://%5b%3a%3affff%3a127.0.0.1%5d/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use a raw IP address"
|
||
|
||
Scenario: base_url validation rejects dot-only hostname
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://./v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must include a valid hostname"
|
||
|
||
Scenario: base_url validation rejects unbalanced bracket in percent-decoded hostname
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://%5b127.0.0.1/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "invalid bracket characters"
|
||
|
||
# ------------------------------------------------------------------
|
||
# m6: _build_native when resolve_class_ref returns None
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: ConfigurationError when resolve_class_ref returns None for a native provider
|
||
Given I construct an LLMAgent for google with credentials dict containing api_key
|
||
And I patch resolve_class_ref to return None
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "is not available for provider"
|
||
|
||
# ------------------------------------------------------------------
|
||
# m4: _execute_multi_actor credential threading
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Executor._execute_multi_actor passes credentials to sub-Executor
|
||
Given an Executor for a multi-actor bundle with openai credentials dict
|
||
When I execute the multi-actor bundle with "Hello from multi-actor"
|
||
Then the multi-actor execution should succeed
|
||
And the sub-Executor should have received the credentials dict
|
||
|
||
# ------------------------------------------------------------------
|
||
# m4: _execute_tool path
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Executor._execute_tool executes tool actor successfully
|
||
Given an Executor for a tool actor
|
||
When I execute the tool actor with "run tool"
|
||
Then the tool execution should succeed with a zero-token result
|
||
|
||
|
||
|
||
|
||
# ------------------------------------------------------------------
|
||
# M1: SSRF Bypass — percent-encoded userinfo (%40) in hostname
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: base_url validation rejects percent-encoded userinfo in hostname
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://user%40127.0.0.1/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not contain userinfo"
|
||
|
||
# ------------------------------------------------------------------
|
||
# m1: _execute_graph ConfigurationError propagation
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Executor._execute_graph propagates ConfigurationError for missing graph agent credentials
|
||
Given an Executor for a graph actor with an empty credentials dict
|
||
When I execute the graph actor expecting ConfigurationError
|
||
Then a credential ConfigurationError should contain "missing credentials for provider"
|
||
|
||
Scenario: Executor._execute_graph propagates AgentCreationError without double-wrapping
|
||
Given an Executor for a graph actor with openai credentials dict
|
||
And I patch AgentFactory.create_agent to raise AgentCreationError
|
||
When I execute the graph actor expecting AgentCreationError
|
||
Then the raised error should be an AgentCreationError containing "Simulated AgentCreationError"
|
||
|
||
# ------------------------------------------------------------------
|
||
# m2: _execute_tool error propagation
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Executor._execute_tool propagates ConfigurationError from tool agent
|
||
Given an Executor for a tool actor
|
||
And I patch ToolAgent.process_message to raise ConfigurationError
|
||
When I execute the tool actor with error test
|
||
Then a credential ConfigurationError should contain "Simulated tool ConfigurationError"
|
||
|
||
Scenario: Executor._execute_tool propagates AgentCreationError from tool agent
|
||
Given an Executor for a tool actor
|
||
And I patch ToolAgent.process_message to raise AgentCreationError
|
||
When I execute the tool actor with error test
|
||
Then the raised error should be an AgentCreationError containing "Simulated tool AgentCreationError"
|
||
|
||
Scenario: Executor._execute_tool wraps RuntimeError in ExecutionError
|
||
Given an Executor for a tool actor
|
||
And I patch ToolAgent.process_message to raise RuntimeError
|
||
When I execute the tool actor for ExecutionError wrapping test
|
||
Then the raised error should be an ExecutionError containing "Tool execution failed"
|
||
|
||
Scenario: Executor._execute_graph wraps RuntimeError in ExecutionError
|
||
Given an Executor for a graph actor with openai credentials dict
|
||
And I patch PureLangGraph.execute to raise RuntimeError
|
||
When I execute the graph actor for ExecutionError wrapping test
|
||
Then the raised error should be an ExecutionError containing "Graph execution failed"
|
||
|
||
# ------------------------------------------------------------------
|
||
# n4: LLMAgent constructor rejects non-str credential inner value
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: LLMAgent constructor rejects non-str credential inner value
|
||
Given I pass a credentials dict with a non-str inner value for "openai".api_key to LLMAgent
|
||
When I construct the LLMAgent with the prepared credentials (expecting error)
|
||
Then a credential ConfigurationError should contain "must be a str"
|
||
|
||
# ------------------------------------------------------------------
|
||
# m4: Temperature override save/restore in process_message
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: process_message restores temperature after _temperature_override
|
||
Given I construct an LLMAgent for openai with credentials dict containing only api_key
|
||
And I inject a mock chat model with temperature 0.5
|
||
When I call process_message with "Hello" and _temperature_override 0.9
|
||
Then the chat_model temperature should be restored to 0.5
|
||
|
||
# ------------------------------------------------------------------
|
||
# m4: Outer except Exception in build_chat_model wraps unexpected
|
||
# non-ConfigurationError exceptions
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: build_chat_model wraps unexpected RuntimeError in ConfigurationError
|
||
Given I construct an LLMAgent for openai with api_key in config and no credentials dict
|
||
And I arrange for ChatOpenAI instantiation to raise RuntimeError
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "Failed to initialize openai model"
|
||
|
||
# ------------------------------------------------------------------
|
||
# m6: LangChainException catch in process_message
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: process_message wraps LangChainException in ExecutionError
|
||
Given I construct an LLMAgent for openai with credentials dict containing only api_key
|
||
And I inject a mock chat model whose ainvoke raises LangChainException
|
||
When I call process_message with "Hello" (expecting error)
|
||
Then the raised error should be an ExecutionError containing "LLM processing failed"
|
||
|
||
# ------------------------------------------------------------------
|
||
# m7: Empty actors dict in _execute_multi_actor
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Executor._execute_multi_actor raises ConfigurationError for empty actors
|
||
Given an Executor for a multi-actor bundle with an empty actors dict
|
||
When I execute the multi-actor bundle with empty actors expecting ConfigurationError
|
||
Then a credential ConfigurationError should contain "Multi-actor bundle has no actors."
|
||
|
||
# ------------------------------------------------------------------
|
||
# m2: _validate_base_url percent-decode iteration cap (for...else guard)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: base_url validation raises ConfigurationError when encoding never stabilizes
|
||
Given I construct an LLMAgent for provider "groq" with base_url "https://api.groq.com/v1" without skipping SSRF validation
|
||
And I patch urllib.parse.unquote to never return a stable hostname
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "encoding did not stabilize"
|
||
|
||
# ------------------------------------------------------------------
|
||
# M2: Duplicate provider key normalization
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: AgentFactory rejects duplicate provider keys after normalization
|
||
Given I pass a credentials dict with "OpenAI" and "openai" keys to AgentFactory
|
||
When I create an AgentFactory with the prepared credentials (expecting error)
|
||
Then a credential ConfigurationError should contain "Duplicate provider after normalization"
|
||
|
||
# ------------------------------------------------------------------
|
||
# M3: SSRF bypass — space (%20) and DEL (%7f) control characters
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: base_url validation rejects percent-encoded space in hostname
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://api%20example.com/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "invalid control characters"
|
||
|
||
Scenario: base_url validation rejects percent-encoded DEL in hostname
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://api%7fexample.com/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "invalid control characters"
|
||
|
||
# ------------------------------------------------------------------
|
||
# m5: Non-string api_key in standalone mode
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Standalone mode raises explicit error for non-string api_key
|
||
Given I construct an LLMAgent for openai without credentials dict and with api_key set to 12345
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "'api_key' must be a string"
|
||
|
||
# ------------------------------------------------------------------
|
||
# n4: __repr__ credential redaction
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: LLMAgent repr does not leak API key
|
||
Given I construct an LLMAgent for openai with credentials dict containing only api_key
|
||
Then the repr of the agent should not contain the api_key value
|
||
|
||
# ------------------------------------------------------------------
|
||
# M2: Inner-dict mutation protection
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Mutating original credentials dict does not affect stored credentials
|
||
Given I construct an LLMAgent for openai with credentials dict containing only api_key
|
||
And I mutate the original credentials dict's api_key to "mutated-key"
|
||
Then the agent's stored api_key should still be the original value
|
||
|
||
# ------------------------------------------------------------------
|
||
# M3: Non-string provider key in validate_credentials_structure
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: AgentFactory constructor rejects non-string provider key
|
||
Given I pass a credentials dict with a non-string provider key to AgentFactory
|
||
When I create an AgentFactory with the prepared credentials (expecting error)
|
||
Then a credential ConfigurationError should contain "credentials keys must be strings"
|
||
|
||
# ------------------------------------------------------------------
|
||
# M4: AgentFactory.create_agent with unknown agent name
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: AgentFactory raises AgentCreationError when asked to create an unknown agent
|
||
Given a minimal actor config with a single openai LLM agent
|
||
And I create an AgentFactory with a credentials dict for openai
|
||
When I call create_agent for an unknown agent named "nonexistent"
|
||
Then the raised error should be an AgentCreationError containing "No configuration found for agent"
|
||
|
||
# ------------------------------------------------------------------
|
||
# M5: Executor.execute unknown actor type
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Executor.execute raises ConfigurationError for unknown actor type
|
||
Given I construct an Executor with actor type "unknown_xyz"
|
||
When I execute the Executor (expecting error)
|
||
Then a ConfigurationError should be raised containing "Cannot execute actor"
|
||
|
||
# ------------------------------------------------------------------
|
||
# M6: ExecutionError propagation in _execute_graph (no double-wrapping)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Executor._execute_graph propagates ExecutionError without double-wrapping
|
||
Given an Executor for a graph actor with openai credentials dict
|
||
And I patch PureLangGraph.execute to raise ExecutionError
|
||
When I execute the graph actor for ExecutionError wrapping test
|
||
Then the raised error should be an ExecutionError containing "Simulated graph ExecutionError"
|
||
|
||
# ------------------------------------------------------------------
|
||
# M7: ExecutionError propagation in _execute_tool (no double-wrapping)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Executor._execute_tool propagates ExecutionError without double-wrapping
|
||
Given an Executor for a tool actor
|
||
And I patch ToolAgent.process_message to raise ExecutionError
|
||
When I execute the tool actor for ExecutionError propagation test
|
||
Then the raised error should be an ExecutionError containing "Simulated tool ExecutionError"
|
||
|
||
# ------------------------------------------------------------------
|
||
# m1: ExecutionError / AgentCreationError propagation in _execute_llm
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Executor._execute_llm propagates ExecutionError without double-wrapping
|
||
Given an Executor for an LLM actor with openai credentials dict
|
||
And I patch LLMAgent.process_message to raise ExecutionError
|
||
When I execute the LLM actor for error propagation test
|
||
Then the raised error should be an ExecutionError containing "Simulated LLM ExecutionError"
|
||
|
||
Scenario: Executor._execute_llm propagates AgentCreationError without double-wrapping
|
||
Given an Executor for an LLM actor with openai credentials dict
|
||
And I patch LLMAgent.process_message to raise AgentCreationError
|
||
When I execute the LLM actor for error propagation test
|
||
Then the raised error should be an AgentCreationError containing "Simulated LLM AgentCreationError"
|
||
|
||
# ------------------------------------------------------------------
|
||
# m2: _execute_graph NodeType ValueError fallback to FUNCTION
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Executor._execute_graph falls back to FUNCTION for invalid node type
|
||
Given an Executor for a graph actor with an invalid node type and openai credentials
|
||
And I record the graph actor config_dict state
|
||
When I execute the graph actor with "Hello from invalid node type"
|
||
Then the graph execution should succeed with the mock graph response
|
||
|
||
# ------------------------------------------------------------------
|
||
# M2: create_agents_from_config credential threading
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: AgentFactory.create_agents_from_config threads credentials to all agents
|
||
Given a multi-agent actor config with openai and anthropic LLM agents
|
||
And I create an AgentFactory with credentials for both providers
|
||
When I call create_agents_from_config
|
||
Then all created LLMAgent instances should have their provider-specific credentials set
|
||
|
||
# ------------------------------------------------------------------
|
||
# m8: URL and hostname length guards
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: base_url validation rejects URL exceeding 2048 characters
|
||
Given I construct an LLMAgent for provider "groq" with a base_url that is 2049 characters long
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "exceeds maximum allowed length"
|
||
|
||
Scenario: base_url validation rejects hostname exceeding 253 characters
|
||
Given I construct an LLMAgent for provider "groq" with a hostname that is 254 characters long
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "exceeds maximum allowed length"
|
||
|
||
# ------------------------------------------------------------------
|
||
# m7: NFKC normalization prevents Unicode homograph localhost bypass
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: base_url validation rejects Unicode homograph of localhost
|
||
Given I construct an LLMAgent for provider "groq" with base_url "https://ℓocalhost/v1" without skipping SSRF validation
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "must not use 'localhost'"
|
||
|
||
# ------------------------------------------------------------------
|
||
# n3: Bare percent character in hostname (IPv6 zone ID)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: base_url validation rejects hostname with bare percent character from IPv6 zone ID
|
||
Given I construct an LLMAgent for provider "groq" with credentials containing api_key and base_url "https://[::1%25]/v1"
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "invalid percent character"
|
||
|
||
# ------------------------------------------------------------------
|
||
# m6: Non-string api_key in standalone mode raises explicit type error
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Standalone mode raises explicit ConfigurationError for non-string api_key
|
||
Given I construct an LLMAgent for openai without credentials dict and with api_key set to 12345
|
||
When I access the chat_model property (expecting error)
|
||
Then a credential ConfigurationError should contain "'api_key' must be a string"
|
||
|
||
# ------------------------------------------------------------------
|
||
# m4: create_executor and Executor accept None credentials
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: create_executor accepts None credentials for standalone mode
|
||
Given a valid LLM actor config for standalone mode
|
||
When I call create_executor with None credentials
|
||
Then the executor should be created successfully with None credentials
|
||
|
||
Scenario: Executor accepts None credentials for standalone mode
|
||
Given a valid LLM actor config for standalone mode
|
||
When I construct an Executor with None credentials
|
||
Then the executor should be created successfully with None credentials
|
||
|
||
# ------------------------------------------------------------------
|
||
# n3: AgentFactory constructor argument validation (factory.py coverage)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: AgentFactory rejects non-dict config argument
|
||
Given I pass a non-dict config "not a dict" to AgentFactory constructor
|
||
When I create an AgentFactory with invalid config (expecting error)
|
||
Then a ConfigurationError should be raised containing "config must be a dict"
|
||
|
||
Scenario: AgentFactory rejects non-TemplateRenderer template_renderer argument
|
||
Given I pass a non-TemplateRenderer "not a renderer" to AgentFactory constructor
|
||
When I create an AgentFactory with invalid template_renderer (expecting error)
|
||
Then a ConfigurationError should be raised containing "template_renderer must be a TemplateRenderer"
|
||
|
||
# ------------------------------------------------------------------
|
||
# n4: validate_credentials_structure paths (llm_providers.py coverage)
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: validate_credentials_structure rejects non-dict non-None credentials via AgentFactory
|
||
Given I pass a non-dict non-None credentials "a string" to AgentFactory via create
|
||
When I create an AgentFactory with the non-dict credentials (expecting error)
|
||
Then a ConfigurationError should be raised containing "credentials must be a dict"
|
||
|
||
Scenario: validate_credentials_structure rejects non-str inner key via AgentFactory
|
||
Given I pass credentials with a non-str inner key to AgentFactory
|
||
When I create an AgentFactory with the non-str inner key credentials (expecting error)
|
||
Then a ConfigurationError should be raised containing "key must be a str"
|
||
|
||
# ------------------------------------------------------------------
|
||
# M1: top-level config overriding conflicting nested config block
|
||
# ------------------------------------------------------------------
|
||
|
||
Scenario: Executor._execute_llm uses top-level model over conflicting nested config model
|
||
Given an LLM actor config with top-level model "gpt-4" and nested config model "gpt-3.5-turbo"
|
||
And credentials dict with openai provider
|
||
When I execute the LLM actor and capture the build_chat_model model argument
|
||
Then the LLM execution should succeed with the mock response
|
||
And the build_chat_model should have received model "gpt-4"
|