Proposal: update specification — diagnostics command now checks all 9 LLM providers #4618

New issue

Open

opened 2026-04-08 17:38:31 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-08 17:38:31 +00:00

Owner

Spec Update Proposal

Triggered by: Merged PR #3469 (fix(cli): extend agents diagnostics to check all 9 supported providers)

What Changed in the Implementation

PR #3469 extended _check_providers() in src/cleveragents/cli/commands/system.py to report diagnostic status for all 9 providers supported by ProviderRegistry. Previously only 4 providers were checked (OpenAI, Anthropic, Google, OpenRouter); now all 9 are covered.

Current implementation (_check_providers()):

provider_checks = [
    ("openai", "OPENAI_API_KEY"),
    ("anthropic", "ANTHROPIC_API_KEY"),
    ("google", "GOOGLE_API_KEY"),
    ("azure", "AZURE_OPENAI_API_KEY"),
    ("openrouter", "OPENROUTER_API_KEY"),
    ("gemini", "GEMINI_API_KEY"),
    ("cohere", "COHERE_API_KEY"),
    ("groq", "GROQ_API_KEY"),
    ("together", "TOGETHER_API_KEY"),
]

What Spec Sections Need Updating

1. agents diagnostics examples (lines 853–1200)

The spec's example output for agents diagnostics only shows 2 provider checks (OPENAI_API_KEY and Anthropic key). The actual output now includes all 9 providers. The example output should be updated to reflect the full provider list.

Current spec example (Rich format, lines 869–899):

$ agents diagnostics

╭─ Checks ────────────────────────────────╮
│ Check            Status  Details        │
│ ───────────────  ──────  ────────────── │
│ Config file      OK      readable       │
│ Database         OK      writable       │
│ OPENAI_API_KEY   WARN    missing        │
│ Anthropic key    OK      configured     │
│ Disk space       OK      2.1 GB free    │
│ Text index       OK      tantivy 0.22   │
│ Vector index     OK      faiss (CPU)    │
│ Graph store      WARN    not configured │
│ File permissions OK      data dir r/w   │
│ Git              OK      git 2.43.0     │
╰─────────────────────────────────────────╯

╭─ Summary ─────────╮
│ Checks: 10 total  │
│ Warnings: 2       │
│ Errors: 0         │
│ Duration: 0.6s    │
╰───────────────────╯

Proposed update (Rich format):

$ agents diagnostics

╭─ Checks ──────────────────────────────────────╮
│ Check              Status  Details             │
│ ─────────────────  ──────  ─────────────────── │
│ Config file        OK      readable            │
│ Database           OK      writable            │
│ Openai key         WARN    missing             │
│ Anthropic key      OK      configured          │
│ Google key         WARN    missing             │
│ Azure key          WARN    missing             │
│ Openrouter key     WARN    missing             │
│ Gemini key         WARN    missing             │
│ Cohere key         WARN    missing             │
│ Groq key           WARN    missing             │
│ Together key       WARN    missing             │
│ Disk space         OK      2.1 GB free         │
│ Text index         OK      tantivy 0.22        │
│ Vector index       OK      faiss (CPU)         │
│ Graph store        WARN    not configured      │
│ File permissions   OK      data dir r/w        │
│ Git                OK      git 2.43.0          │
╰───────────────────────────────────────────────╯

╭─ Summary ──────────╮
│ Checks: 17 total   │
│ Warnings: 9        │
│ Errors: 0          │
│ Duration: 0.6s     │
╰────────────────────╯

2. Provider configuration reference table (lines 30805–30818)

The spec's provider.* configuration table is missing three providers that the implementation now supports:

cohere — COHERE_API_KEY
groq — GROQ_API_KEY
together — TOGETHER_API_KEY

Proposed additions to the table:

Key	Type	Default	Env Variable	Description
`provider.cohere.api-key`	string	(not set)	`COHERE_API_KEY`	API key for Cohere models. Required when using Cohere-based actors.
`provider.groq.api-key`	string	(not set)	`GROQ_API_KEY`	API key for Groq. Required when using Groq-hosted models.
`provider.together.api-key`	string	(not set)	`TOGETHER_API_KEY`	API key for Together AI. Required when using Together AI-hosted models.

Rationale

The implementation correctly extended provider coverage to match the full ProviderRegistry support. Users of Groq, Together AI, Cohere, Azure, or Gemini previously received no diagnostic feedback about their provider configuration. The spec examples and config reference table should reflect the actual behavior.

This is a major spec change (updates CLI output examples and configuration reference) and requires human approval before being committed.

Automated by CleverAgents Bot
Supervisor: Spec Evolution | Agent: spec-updater

## Spec Update Proposal **Triggered by:** Merged PR #3469 (`fix(cli): extend agents diagnostics to check all 9 supported providers`) ### What Changed in the Implementation PR #3469 extended `_check_providers()` in `src/cleveragents/cli/commands/system.py` to report diagnostic status for all 9 providers supported by `ProviderRegistry`. Previously only 4 providers were checked (OpenAI, Anthropic, Google, OpenRouter); now all 9 are covered. **Current implementation (`_check_providers()`):** ```python provider_checks = [ ("openai", "OPENAI_API_KEY"), ("anthropic", "ANTHROPIC_API_KEY"), ("google", "GOOGLE_API_KEY"), ("azure", "AZURE_OPENAI_API_KEY"), ("openrouter", "OPENROUTER_API_KEY"), ("gemini", "GEMINI_API_KEY"), ("cohere", "COHERE_API_KEY"), ("groq", "GROQ_API_KEY"), ("together", "TOGETHER_API_KEY"), ] ``` ### What Spec Sections Need Updating **1. `agents diagnostics` examples (lines 853–1200)** The spec's example output for `agents diagnostics` only shows 2 provider checks (OPENAI_API_KEY and Anthropic key). The actual output now includes all 9 providers. The example output should be updated to reflect the full provider list. **Current spec example (Rich format, lines 869–899):** ``` $ agents diagnostics ╭─ Checks ────────────────────────────────╮ │ Check Status Details │ │ ─────────────── ────── ────────────── │ │ Config file OK readable │ │ Database OK writable │ │ OPENAI_API_KEY WARN missing │ │ Anthropic key OK configured │ │ Disk space OK 2.1 GB free │ │ Text index OK tantivy 0.22 │ │ Vector index OK faiss (CPU) │ │ Graph store WARN not configured │ │ File permissions OK data dir r/w │ │ Git OK git 2.43.0 │ ╰─────────────────────────────────────────╯ ╭─ Summary ─────────╮ │ Checks: 10 total │ │ Warnings: 2 │ │ Errors: 0 │ │ Duration: 0.6s │ ╰───────────────────╯ ``` **Proposed update (Rich format):** ``` $ agents diagnostics ╭─ Checks ──────────────────────────────────────╮ │ Check Status Details │ │ ───────────────── ────── ─────────────────── │ │ Config file OK readable │ │ Database OK writable │ │ Openai key WARN missing │ │ Anthropic key OK configured │ │ Google key WARN missing │ │ Azure key WARN missing │ │ Openrouter key WARN missing │ │ Gemini key WARN missing │ │ Cohere key WARN missing │ │ Groq key WARN missing │ │ Together key WARN missing │ │ Disk space OK 2.1 GB free │ │ Text index OK tantivy 0.22 │ │ Vector index OK faiss (CPU) │ │ Graph store WARN not configured │ │ File permissions OK data dir r/w │ │ Git OK git 2.43.0 │ ╰───────────────────────────────────────────────╯ ╭─ Summary ──────────╮ │ Checks: 17 total │ │ Warnings: 9 │ │ Errors: 0 │ │ Duration: 0.6s │ ╰────────────────────╯ ``` **2. Provider configuration reference table (lines 30805–30818)** The spec's `provider.*` configuration table is missing three providers that the implementation now supports: - `cohere` — `COHERE_API_KEY` - `groq` — `GROQ_API_KEY` - `together` — `TOGETHER_API_KEY` **Proposed additions to the table:** | Key | Type | Default | Env Variable | Description | |-----|------|---------|-------------|-------------| | `provider.cohere.api-key` | string | *(not set)* | `COHERE_API_KEY` | API key for Cohere models. Required when using Cohere-based actors. | | `provider.groq.api-key` | string | *(not set)* | `GROQ_API_KEY` | API key for Groq. Required when using Groq-hosted models. | | `provider.together.api-key` | string | *(not set)* | `TOGETHER_API_KEY` | API key for Together AI. Required when using Together AI-hosted models. | ### Rationale The implementation correctly extended provider coverage to match the full `ProviderRegistry` support. Users of Groq, Together AI, Cohere, Azure, or Gemini previously received no diagnostic feedback about their provider configuration. The spec examples and config reference table should reflect the actual behavior. This is a **major spec change** (updates CLI output examples and configuration reference) and requires human approval before being committed. --- **Automated by CleverAgents Bot** Supervisor: Spec Evolution | Agent: spec-updater

HAL9000 referenced this issue

2026-04-08 18:43:13 +00:00

[Automated] CleverAgents Build Session - 2026-04-08 #4743

HAL9000 referenced this issue

2026-04-08 19:33:22 +00:00

[Automated] CleverAgents Build Session - 2026-04-08 #4799

freemo added the

Rows
Columns