cleveragents/cleveragents-core
4
0
Fork 3
Code Issues 6.0k Pull Requests 115 Actions Packages Projects Releases Wiki Activity

[TEST-INFRA] Add Robot framework integration tests for the provider module (#10952) #10977

Open
HAL9000 wants to merge 6 commits from feature/issue-10952-provider-integration-tests into master
pull from: feature/issue-10952-provider-integration-tests
merge into: cleveragents:master
Conversation 12 Commits 6 Files Changed 11
+1470 -8

6 Commits

Author SHA1 Message Date
HAL9000 0df56da142 fix(tests): isolate provider env vars and add langchain-groq dep in integration tests
CI / lint (pull_request) Successful in 38s
Details
CI / push-validation (pull_request) Successful in 31s
Details
CI / build (pull_request) Successful in 46s
Details
CI / typecheck (pull_request) Successful in 1m4s
Details
CI / quality (pull_request) Successful in 1m6s
Details
CI / security (pull_request) Successful in 1m12s
Details
CI / unit_tests (pull_request) Successful in 6m40s
Details
CI / docker (pull_request) Successful in 1m34s
Details
CI / integration_tests (pull_request) Successful in 8m36s
Details
CI / helm (pull_request) Failing after 13m41s
Details
CI / coverage (pull_request) Successful in 11m58s
Details
CI / status-check (pull_request) Failing after 3s
Details 
8 Robot Framework provider integration tests were failing because host
environment API keys (e.g. OPENAI_API_KEY) leaked into the test subprocess,
causing is_provider_configured() to return True when the tests expected False,
and GROQ provider tests failed due to missing langchain-groq package.

- Add _save_clear_provider_env()/_restore_provider_env() helpers to
  robot/helper_provider_integration.py that scrub all known provider env
  vars before each test and restore them in a finally block
- Apply env isolation to the 7 tests that check unconfigured-provider
  behaviour: is_provider_configured_{true,false},
  get_default_provider_type_{env_override,fallback_order,invalid_env_ignored,
  unconfigured_env_skipped}, resolve_provider_by_name_unconfigured_raises
- Add langchain-groq>=0.1.0 to pyproject.toml so GROQ provider tests can
  import ChatGroq without ModuleNotFoundError

ISSUES CLOSED: #10952
 2026-06-10 10:39:51 -04:00
HAL9000 c64132da14 fix(lint): resolve 83 ruff errors in provider integration test helper 
- Remove unused ProviderCapabilities import from test_default_capabilities_openai
- Replace 7x assert False with raise AssertionError (B011)
- Replace setattr(caps, ...) with direct assignment (B010)
- Rewrite dispatch table as single-line entries with # noqa: E501 on
  lines exceeding 88 chars (fixes ruff format --check mismatch)
- Collapse two short assert messages from multi-line to single-line
- Fix robot helper path: ${HELPER} is absolute, drop robot/ prefix
- Fix CONTRIBUTORS.md: correct PR number (#10977) and test count (43)

ISSUES CLOSED: #10952
 2026-06-10 10:39:51 -04:00
HAL9000 5c5cff8ee2 test(infra): add Robot framework integration tests for provider module (#10952) 
Add comprehensive Robot Framework integration test suite for the provider
module with 43 test cases covering: ProviderRegistry discovery, configured
provider filtering, provider info lookups by enum/string/uppercase, default
provider/model resolution with env var overrides, DEFAULT_CAPABILITIES and
DEFAULT_MODELS verification for all 10 provider types, FALLBACK_ORDER order,
global singleton/reset behavior, AI provider creation (including MOCK sandboxing),
LLM factory instantiation, named provider resolution with error handling,
ProviderCapabilities immutability, and Azure provider config fields.

Also update CHANGELOG.md and CONTRIBUTORS.md.

ISSUES CLOSED: #10952
 2026-06-10 10:39:51 -04:00
clever-agent 34cb1c3b75 build: implement explicit-vs-fetched variable propagation across all agents 
All agents now track which variables were explicitly present in their prompt
versus fetched from environment variables or git remote. When constructing
subagent prompts, only explicitly-present variables are included. Fetched
variables are omitted, allowing each subagent to fetch them independently.

This prevents credentials and other fetched values from being garbled as they
propagate through multiple LLM prompt layers.

Affected agents:
- auto-agents (primary orchestrator)
- implementation-supervisor, pr-merge-supervisor, pr-review-supervisor
- supervisor (generic)
- implementation-worker, pr-merge-worker, pr-review-worker
- task-implementor, tier-dispatcher
- work-group-util, git-clone-util, git-push-util, git-checkout-util
 2026-06-10 10:39:51 -04:00
clever-agent ac64d5f03e build: added critical rule clearly stating no question asking 2026-06-10 10:39:51 -04:00
clever-agent 4d80765571 build: pulled in new endpoints and fixing permission from experimental branch 2026-06-10 10:39:51 -04:00