ops(ci): Configure LLM API keys in Forgejo CI for integration test execution #701

Closed
opened 2026-03-11 19:02:10 +00:00 by freemo · 1 comment
Owner

Summary

Integration tests must exercise real LLM endpoints (per CONTRIBUTING.md — mocking is prohibited in integration tests). The Forgejo CI pipeline currently has no LLM API keys configured, which means integration tests that call real LLM endpoints will fail in CI. This ticket adds the necessary secrets and pipeline configuration.

Background

After removing all mocks from Robot Framework integration tests (#698, #699, #700), the integration test suite will require real API keys to pass. Currently:

  • No ANTHROPIC_API_KEY secret exists in Forgejo CI
  • No OPENAI_API_KEY secret exists in Forgejo CI
  • The nox sessions for integration tests do not inject these environment variables

Metadata

  • Commit Message: ops(ci): configure LLM API keys in Forgejo CI for integration test execution
  • Branch: ops/m3-ci-llm-keys

Acceptance Criteria

  • Anthropic API key is stored as a Forgejo CI secret (not in source code)
  • OpenAI API key is stored as a Forgejo CI secret (not in source code)
  • The CI pipeline injects ANTHROPIC_API_KEY and OPENAI_API_KEY as environment variables during Robot Framework integration test runs
  • At least one integration test that calls a real LLM endpoint passes in CI
  • API keys are NOT exposed in CI logs or build artifacts
  • Unit tests (Behave BDD) continue to run without requiring API keys

Subtasks

  • Add ANTHROPIC_API_KEY as a Forgejo repository or organization secret
  • Add OPENAI_API_KEY as a Forgejo repository or organization secret
  • Update CI pipeline configuration to inject secrets as environment variables during integration test jobs
  • Verify secrets are masked in CI output logs
  • Run a CI build with integration tests to confirm at least one real-LLM test passes
  • Document the secret names and configuration in the project's CI documentation or README

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
## Summary Integration tests must exercise real LLM endpoints (per CONTRIBUTING.md — mocking is prohibited in integration tests). The Forgejo CI pipeline currently has no LLM API keys configured, which means integration tests that call real LLM endpoints will fail in CI. This ticket adds the necessary secrets and pipeline configuration. ## Background After removing all mocks from Robot Framework integration tests (#698, #699, #700), the integration test suite will require real API keys to pass. Currently: - No `ANTHROPIC_API_KEY` secret exists in Forgejo CI - No `OPENAI_API_KEY` secret exists in Forgejo CI - The nox sessions for integration tests do not inject these environment variables ## Metadata - **Commit Message**: `ops(ci): configure LLM API keys in Forgejo CI for integration test execution` - **Branch**: `ops/m3-ci-llm-keys` ## Acceptance Criteria - [ ] Anthropic API key is stored as a Forgejo CI secret (not in source code) - [ ] OpenAI API key is stored as a Forgejo CI secret (not in source code) - [ ] The CI pipeline injects `ANTHROPIC_API_KEY` and `OPENAI_API_KEY` as environment variables during Robot Framework integration test runs - [ ] At least one integration test that calls a real LLM endpoint passes in CI - [ ] API keys are NOT exposed in CI logs or build artifacts - [ ] Unit tests (Behave BDD) continue to run without requiring API keys ## Subtasks - [ ] Add `ANTHROPIC_API_KEY` as a Forgejo repository or organization secret - [ ] Add `OPENAI_API_KEY` as a Forgejo repository or organization secret - [ ] Update CI pipeline configuration to inject secrets as environment variables during integration test jobs - [ ] Verify secrets are masked in CI output logs - [ ] Run a CI build with integration tests to confirm at least one real-LLM test passes - [ ] Document the secret names and configuration in the project's CI documentation or README ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done.
freemo added this to the v3.2.0 milestone 2026-03-11 19:03:59 +00:00
freemo self-assigned this 2026-03-11 20:17:32 +00:00
Author
Owner

completed

completed
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#701
No description provided.