From dddfa48eb4914eb06090804ff8d9d2ed48a90883 Mon Sep 17 00:00:00 2001 From: CleverThis Date: Thu, 9 Apr 2026 06:26:51 +0000 Subject: [PATCH 1/2] docs(spec): extend diagnostics example to cover all providers - update agents diagnostics examples to show all provider checks across rich, plain, JSON, and YAML formats - refresh error scenario samples to reflect the expanded provider coverage and recommendation counts - align CLI diagnostics output with the spec by adding explicit provider display names and consistent ordering Closes #5320 --- docs/specification.md | 306 ++++++++++++++++-------- src/cleveragents/cli/commands/system.py | 26 +- 2 files changed, 221 insertions(+), 111 deletions(-) diff --git a/docs/specification.md b/docs/specification.md index be1d40816..f7c1a9cc4 100644 --- a/docs/specification.md +++ b/docs/specification.md @@ -868,35 +868,48 @@ None. 

     $ agents diagnostics
 
-    ╭─ Checks ────────────────────────────────╮
-    │ Check            Status  Details        │
-    │ ───────────────  ──────  ────────────── │
-    │ Config file      OK      readable       │
-    │ Database         OK      writable       │
-    │ OPENAI_API_KEY   WARN    missing        │
-    │ Anthropic key    OK      configured     │
-    │ Disk space       OK      2.1 GB free    │
-    │ Text index       OK      tantivy 0.22   │
-    │ Vector index     OK      faiss (CPU)    │
-    │ Graph store      WARN    not configured │
-    │ File permissions OK      data dir r/w   │
-    │ Git              OK      git 2.43.0     │
-    ╰─────────────────────────────────────────╯
+    ╭─ Checks ──────────────────────────────────────────────────╮
+    │ Check                     Status  Details               │
+    │ ───────────────────────────  ──────  ───────────────────── │
+    │ Config file               OK      readable               │
+    │ Database                  OK      writable               │
+    │ OpenAI key                WARN    missing                │
+    │ Anthropic key             OK      configured             │
+    │ Google key                WARN    missing                │
+    │ Gemini key                WARN    missing                │
+    │ Azure OpenAI key          WARN    missing                │
+    │ OpenRouter key            WARN    missing                │
+    │ Cohere key                WARN    missing                │
+    │ Groq key                  WARN    missing                │
+    │ Together key              WARN    missing                │
+    │ Disk space                OK      2.1 GB free            │
+    │ Text index                OK      tantivy 0.22           │
+    │ Vector index              OK      faiss (CPU)            │
+    │ Graph store               WARN    not configured         │
+    │ File permissions          OK      data dir r/w           │
+    │ Git                       OK      git 2.43.0             │
+    ╰───────────────────────────────────────────────────────────╯
 
     ╭─ Summary ─────────╮
-    │ Checks: 10 total  │
-    │ Warnings: 2       │
+    │ Checks: 17 total  │
+    │ Warnings: 9       │
     │ Errors: 0         │
     │ Duration: 0.6s    │
     ╰───────────────────╯
 
-    ╭─ Recommendations ─────────────────────────────────────────────╮
-    │ - Set OPENAI_API_KEY to enable OpenAI models                  │
-    │ - Configure a graph store backend for structural code queries │
-    │ - Verify provider credentials via config                      │
-    ╰───────────────────────────────────────────────────────────────╯
+    ╭─ Recommendations ──────────────────────────────────────────────╮
+    │ - Set OPENAI_API_KEY to enable OpenAI models                   │
+    │ - Set GOOGLE_API_KEY to enable Google models                   │
+    │ - Set GEMINI_API_KEY to enable Gemini models                   │
+    │ - Set AZURE_OPENAI_API_KEY to enable Azure OpenAI models       │
+    │ - Set OPENROUTER_API_KEY to enable OpenRouter models           │
+    │ - Set COHERE_API_KEY to enable Cohere models                   │
+    │ - Set GROQ_API_KEY to enable Groq models                       │
+    │ - Set TOGETHER_API_KEY to enable Together models               │
+    │ - Configure a graph store backend for structural code queries  │
+    ╰────────────────────────────────────────────────────────────────╯
 
-    ⚠ WARN 2 warnings require attention
+    ⚠ WARN 9 warnings require attention
=== "Plain" @@ -905,31 +918,44 @@ None. $ agents diagnostics Checks - Check Status Details - --------------- ------ -------------- - Config file OK readable - Database OK writable - OPENAI_API_KEY WARN missing - Anthropic key OK configured - Disk space OK 2.1 GB free - Text index OK tantivy 0.22 - Vector index OK faiss (CPU) - Graph store WARN not configured - File permissions OK data dir r/w - Git OK git 2.43.0 + Check Status Details + ------------------------ ------ -------------- + Config file OK readable + Database OK writable + OpenAI key WARN missing + Anthropic key OK configured + Google key WARN missing + Gemini key WARN missing + Azure OpenAI key WARN missing + OpenRouter key WARN missing + Cohere key WARN missing + Groq key WARN missing + Together key WARN missing + Disk space OK 2.1 GB free + Text index OK tantivy 0.22 + Vector index OK faiss (CPU) + Graph store WARN not configured + File permissions OK data dir r/w + Git OK git 2.43.0 Summary - Checks: 10 total - Warnings: 2 + Checks: 17 total + Warnings: 9 Errors: 0 Duration: 0.6s Recommendations - Set OPENAI_API_KEY to enable OpenAI models + - Set GOOGLE_API_KEY to enable Google models + - Set GEMINI_API_KEY to enable Gemini models + - Set AZURE_OPENAI_API_KEY to enable Azure OpenAI models + - Set OPENROUTER_API_KEY to enable OpenRouter models + - Set COHERE_API_KEY to enable Cohere models + - Set GROQ_API_KEY to enable Groq models + - Set TOGETHER_API_KEY to enable Together models - Configure a graph store backend for structural code queries - - Verify provider credentials via config - [WARN] 2 warnings require attention + [WARN] 9 warnings require attention ``` === "JSON" @@ -943,8 +969,15 @@ None. "checks": [ { "check": "Config file", "status": "ok", "details": "readable" }, { "check": "Database", "status": "ok", "details": "writable" }, - { "check": "OPENAI_API_KEY", "status": "warn", "details": "missing" }, + { "check": "OpenAI key", "status": "warn", "details": "missing" }, { "check": "Anthropic key", "status": "ok", "details": "configured" }, + { "check": "Google key", "status": "warn", "details": "missing" }, + { "check": "Gemini key", "status": "warn", "details": "missing" }, + { "check": "Azure OpenAI key", "status": "warn", "details": "missing" }, + { "check": "OpenRouter key", "status": "warn", "details": "missing" }, + { "check": "Cohere key", "status": "warn", "details": "missing" }, + { "check": "Groq key", "status": "warn", "details": "missing" }, + { "check": "Together key", "status": "warn", "details": "missing" }, { "check": "Disk space", "status": "ok", "details": "2.1 GB free" }, { "check": "Text index", "status": "ok", "details": "tantivy 0.22" }, { "check": "Vector index", "status": "ok", "details": "faiss (CPU)" }, @@ -953,22 +986,28 @@ None. { "check": "Git", "status": "ok", "details": "git 2.43.0" } ], "summary": { - "total": 10, - "warnings": 2, + "total": 17, + "warnings": 9, "errors": 0, "duration_s": 0.6 }, "recommendations": [ "Set OPENAI_API_KEY to enable OpenAI models", - "Configure a graph store backend for structural code queries", - "Verify provider credentials via config" + "Set GOOGLE_API_KEY to enable Google models", + "Set GEMINI_API_KEY to enable Gemini models", + "Set AZURE_OPENAI_API_KEY to enable Azure OpenAI models", + "Set OPENROUTER_API_KEY to enable OpenRouter models", + "Set COHERE_API_KEY to enable Cohere models", + "Set GROQ_API_KEY to enable Groq models", + "Set TOGETHER_API_KEY to enable Together models", + "Configure a graph store backend for structural code queries" ] }, "timing": { "duration_ms": 600 }, "messages": [ - { "level": "warn", "text": "2 warnings require attention" } + { "level": "warn", "text": "9 warnings require attention" } ] } ``` @@ -987,12 +1026,33 @@ None. - check: Database status: ok details: writable - - check: OPENAI_API_KEY + - check: OpenAI key status: warn details: missing - check: Anthropic key status: ok details: configured + - check: Google key + status: warn + details: missing + - check: Gemini key + status: warn + details: missing + - check: Azure OpenAI key + status: warn + details: missing + - check: OpenRouter key + status: warn + details: missing + - check: Cohere key + status: warn + details: missing + - check: Groq key + status: warn + details: missing + - check: Together key + status: warn + details: missing - check: Disk space status: ok details: 2.1 GB free @@ -1012,19 +1072,25 @@ None. status: ok details: git 2.43.0 summary: - total: 10 - warnings: 2 + total: 17 + warnings: 9 errors: 0 duration_s: 0.6 recommendations: - Set OPENAI_API_KEY to enable OpenAI models + - Set GOOGLE_API_KEY to enable Google models + - Set GEMINI_API_KEY to enable Gemini models + - Set AZURE_OPENAI_API_KEY to enable Azure OpenAI models + - Set OPENROUTER_API_KEY to enable OpenRouter models + - Set COHERE_API_KEY to enable Cohere models + - Set GROQ_API_KEY to enable Groq models + - Set TOGETHER_API_KEY to enable Together models - Configure a graph store backend for structural code queries - - Verify provider credentials via config timing: duration_ms: 600 messages: - level: warn - text: 2 warnings require attention + text: 9 warnings require attention ``` When critical checks fail, diagnostics reports errors: @@ -1034,24 +1100,31 @@ When critical checks fail, diagnostics reports errors: 

     $ agents diagnostics
 
-    ╭─ Checks ────────────────────────────────────────────╮
-    │ Check            Status   Details                   │
-    │ ───────────────  ───────  ────────────────────      │
-    │ Config file      OK       readable                  │
-    │ Database         ERROR    locked by another process │
-    │ OPENAI_API_KEY   OK       configured                │
-    │ Anthropic key    ERROR    invalid key format        │
-    │ Disk space       WARN     312 MB free (low)         │
-    │ Text index       OK       tantivy 0.22              │
-    │ Vector index     ERROR    FAISS library not found   │
-    │ Graph store      OK       neo4j 5.15                │
-    │ File permissions OK       data dir r/w              │
-    │ Git              OK       git 2.43.0                │
-    ╰─────────────────────────────────────────────────────╯
+    ╭─ Checks ──────────────────────────────────────────────────╮
+    │ Check                     Status   Details                   │
+    │ ───────────────────────────  ───────  ───────────────────────────── │
+    │ Config file               OK       readable                    │
+    │ Database                  ERROR    locked by another process    │
+    │ OpenAI key                ERROR    invalid key format           │
+    │ Anthropic key             OK       configured                  │
+    │ Google key                WARN     missing                     │
+    │ Gemini key                WARN     missing                     │
+    │ Azure OpenAI key          WARN     missing                     │
+    │ OpenRouter key            WARN     missing                     │
+    │ Cohere key                WARN     missing                     │
+    │ Groq key                  WARN     missing                     │
+    │ Together key              WARN     missing                     │
+    │ Disk space                WARN     312 MB free (low)            │
+    │ Text index                OK       tantivy 0.22                │
+    │ Vector index              ERROR    FAISS library not found      │
+    │ Graph store               OK       neo4j 5.15                   │
+    │ File permissions          OK       data dir r/w                 │
+    │ Git                       OK       git 2.43.0                  │
+    ╰───────────────────────────────────────────────────────────╯
 
     ╭─ Summary ─────────╮
-    │ Checks: 10 total  │
-    │ Warnings: 1       │
+    │ Checks: 17 total  │
+    │ Warnings: 8       │
     │ Errors: 3         │
     │ Duration: 1.2s    │
     ╰───────────────────╯
@@ -1059,8 +1132,8 @@ When critical checks fail, diagnostics reports errors:
     ╭─ Errors (must fix) ─────────────────────────────────────────────────╮
     │ 1. Database is locked by PID 12847 — stop the other process or      │
     │    delete the lock file at ~/.cleveragents/agents.db-lock           │
-    │ 2. Anthropic key starts with "pk-" — expected "sk-ant-" prefix      │
-    │    Run: agents config set provider.anthropic.api-key                │
+    │ 2. OpenAI key starts with "pk-" — expected "sk-" prefix          │
+    │    Run: agents config set provider.openai.api-key      │
     │ 3. FAISS library not installed — vector search will not work        │
     │    Run: pip install faiss-cpu                                       │
     ╰─────────────────────────────────────────────────────────────────────╯
@@ -1074,30 +1147,37 @@ When critical checks fail, diagnostics reports errors:
     $ agents diagnostics
 
     Checks
-      Check            Status   Details
-      ---------------  -------  ------------------------
-      Config file      OK       readable
-      Database         ERROR    locked by another process
-      OPENAI_API_KEY   OK       configured
-      Anthropic key    ERROR    invalid key format
-      Disk space       WARN     312 MB free (low)
-      Text index       OK       tantivy 0.22
-      Vector index     ERROR    FAISS library not found
-      Graph store      OK       neo4j 5.15
-      File permissions OK       data dir r/w
-      Git              OK       git 2.43.0
+      Check                     Status   Details
+      ------------------------  -------  --------------------------
+      Config file               OK       readable
+      Database                  ERROR    locked by another process
+      OpenAI key                ERROR    invalid key format
+      Anthropic key             OK       configured
+      Google key                WARN     missing
+      Gemini key                WARN     missing
+      Azure OpenAI key          WARN     missing
+      OpenRouter key            WARN     missing
+      Cohere key                WARN     missing
+      Groq key                  WARN     missing
+      Together key              WARN     missing
+      Disk space                WARN     312 MB free (low)
+      Text index                OK       tantivy 0.22
+      Vector index              ERROR    FAISS library not found
+      Graph store               OK       neo4j 5.15
+      File permissions          OK       data dir r/w
+      Git                       OK       git 2.43.0
 
     Summary
-      Checks: 10 total
-      Warnings: 1
+      Checks: 17 total
+      Warnings: 8
       Errors: 3
       Duration: 1.2s
 
     Errors (must fix)
       1. Database is locked by PID 12847 -- stop the other process or
          delete the lock file at ~/.cleveragents/agents.db-lock
-      2. Anthropic key starts with "pk-" -- expected "sk-ant-" prefix
-         Run: agents config set provider.anthropic.api-key 
+      2. OpenAI key starts with "pk-" -- expected "sk-" prefix
+         Run: agents config set provider.openai.api-key 
       3. FAISS library not installed -- vector search will not work
          Run: pip install faiss-cpu
 
@@ -1115,8 +1195,15 @@ When critical checks fail, diagnostics reports errors:
         "checks": [
           { "check": "Config file", "status": "ok", "details": "readable" },
           { "check": "Database", "status": "error", "details": "locked by another process" },
-          { "check": "OPENAI_API_KEY", "status": "ok", "details": "configured" },
-          { "check": "Anthropic key", "status": "error", "details": "invalid key format" },
+          { "check": "OpenAI key", "status": "error", "details": "invalid key format" },
+          { "check": "Anthropic key", "status": "ok", "details": "configured" },
+          { "check": "Google key", "status": "warn", "details": "missing" },
+          { "check": "Gemini key", "status": "warn", "details": "missing" },
+          { "check": "Azure OpenAI key", "status": "warn", "details": "missing" },
+          { "check": "OpenRouter key", "status": "warn", "details": "missing" },
+          { "check": "Cohere key", "status": "warn", "details": "missing" },
+          { "check": "Groq key", "status": "warn", "details": "missing" },
+          { "check": "Together key", "status": "warn", "details": "missing" },
           { "check": "Disk space", "status": "warn", "details": "312 MB free (low)" },
           { "check": "Text index", "status": "ok", "details": "tantivy 0.22" },
           { "check": "Vector index", "status": "error", "details": "FAISS library not found" },
@@ -1125,8 +1212,8 @@ When critical checks fail, diagnostics reports errors:
           { "check": "Git", "status": "ok", "details": "git 2.43.0" }
         ],
         "summary": {
-          "total": 10,
-          "warnings": 1,
+          "total": 17,
+          "warnings": 8,
           "errors": 3,
           "duration_s": 1.2
         },
@@ -1137,8 +1224,8 @@ When critical checks fail, diagnostics reports errors:
           },
           {
             "index": 2,
-            "message": "Anthropic key starts with \"pk-\" -- expected \"sk-ant-\" prefix",
-            "fix": "agents config set provider.anthropic.api-key "
+            "message": "OpenAI key starts with \"pk-\" -- expected \"sk-\" prefix",
+            "fix": "agents config set provider.openai.api-key "
           },
           {
             "index": 3,
@@ -1170,12 +1257,33 @@ When critical checks fail, diagnostics reports errors:
         - check: Database
           status: error
           details: locked by another process
-        - check: OPENAI_API_KEY
-          status: ok
-          details: configured
-        - check: Anthropic key
+        - check: OpenAI key
           status: error
           details: invalid key format
+        - check: Anthropic key
+          status: ok
+          details: configured
+        - check: Google key
+          status: warn
+          details: missing
+        - check: Gemini key
+          status: warn
+          details: missing
+        - check: Azure OpenAI key
+          status: warn
+          details: missing
+        - check: OpenRouter key
+          status: warn
+          details: missing
+        - check: Cohere key
+          status: warn
+          details: missing
+        - check: Groq key
+          status: warn
+          details: missing
+        - check: Together key
+          status: warn
+          details: missing
         - check: Disk space
           status: warn
           details: 312 MB free (low)
@@ -1195,8 +1303,8 @@ When critical checks fail, diagnostics reports errors:
           status: ok
           details: git 2.43.0
       summary:
-        total: 10
-        warnings: 1
+        total: 17
+        warnings: 8
         errors: 3
         duration_s: 1.2
       errors:
@@ -1205,10 +1313,12 @@ When critical checks fail, diagnostics reports errors:
             Database is locked by PID 12847 -- stop the other process or
             delete the lock file at ~/.cleveragents/agents.db-lock
         - index: 2
-          message: Anthropic key starts with "pk-" -- expected "sk-ant-" prefix
-          fix: agents config set provider.anthropic.api-key 
+          message: >-
+            OpenAI key starts with "pk-" -- expected "sk-" prefix
+          fix: agents config set provider.openai.api-key 
         - index: 3
-          message: FAISS library not installed -- vector search will not work
+          message: >-
+            FAISS library not installed -- vector search will not work
           fix: pip install faiss-cpu
     timing:
       duration_ms: 1200
diff --git a/src/cleveragents/cli/commands/system.py b/src/cleveragents/cli/commands/system.py
index b1416adf1..42bba1fe5 100644
--- a/src/cleveragents/cli/commands/system.py
+++ b/src/cleveragents/cli/commands/system.py
@@ -253,26 +253,26 @@ def _check_providers() -> list[dict[str, Any]]:
     results: list[dict[str, Any]] = []
 
     provider_checks = [
-        ("openai", "OPENAI_API_KEY"),
-        ("anthropic", "ANTHROPIC_API_KEY"),
-        ("google", "GOOGLE_API_KEY"),
-        ("azure", "AZURE_OPENAI_API_KEY"),
-        ("openrouter", "OPENROUTER_API_KEY"),
-        ("gemini", "GEMINI_API_KEY"),
-        ("cohere", "COHERE_API_KEY"),
-        ("groq", "GROQ_API_KEY"),
-        ("together", "TOGETHER_API_KEY"),
+        ("openai", "OpenAI", "OPENAI_API_KEY"),
+        ("anthropic", "Anthropic", "ANTHROPIC_API_KEY"),
+        ("google", "Google", "GOOGLE_API_KEY"),
+        ("gemini", "Gemini", "GEMINI_API_KEY"),
+        ("azure", "Azure OpenAI", "AZURE_OPENAI_API_KEY"),
+        ("openrouter", "OpenRouter", "OPENROUTER_API_KEY"),
+        ("cohere", "Cohere", "COHERE_API_KEY"),
+        ("groq", "Groq", "GROQ_API_KEY"),
+        ("together", "Together", "TOGETHER_API_KEY"),
     ]
 
-    for provider_name, env_var in provider_checks:
-        configured = settings.has_provider_configured(provider_name)
+    for provider_slug, display_name, env_var in provider_checks:
+        configured = settings.has_provider_configured(provider_slug)
         results.append(
             {
-                "name": f"{provider_name.capitalize()} key",
+                "name": f"{display_name} key",
                 "status": CheckStatus.OK if configured else CheckStatus.WARN,
                 "details": "configured" if configured else "missing",
                 "recommendation": (
-                    f"Set {env_var} to enable {provider_name.capitalize()} models"
+                    f"Set {env_var} to enable {display_name} models"
                     if not configured
                     else None
                 ),
-- 
2.52.0


From 05fbf99b1f97076fb5a890da56bb825eb69069b1 Mon Sep 17 00:00:00 2001
From: HAL9000 
Date: Thu, 23 Apr 2026 09:30:50 +0000
Subject: [PATCH 2/2] docs(spec): extend agents diagnostics example to show all
 9 supported providers

Add CHANGELOG entry documenting the expansion of agents diagnostics
spec examples to cover all 9 supported providers (OpenAI, Anthropic,
Google, Gemini, Azure, OpenRouter, Cohere, Groq, Together), matching
the implementation from PR #3469.

ISSUES CLOSED: #5320
---
 CHANGELOG.md | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/CHANGELOG.md b/CHANGELOG.md
index 24e7b55ad..64841acd4 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -5,6 +5,15 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 
 ## [Unreleased]
 
+### Changed
+
+- **Diagnostics spec examples expanded to all 9 providers** (#5320): Updated the
+  `agents diagnostics` command examples in the specification to show all 9 supported
+  providers (OpenAI, Anthropic, Google, Gemini, Azure, OpenRouter, Cohere, Groq,
+  Together), matching the implementation from PR #3469. Rich, plain, JSON, and YAML
+  example outputs now reflect comprehensive provider coverage with accurate warning
+  counts and per-provider recommendations.
+
 ### Added
 
 - **Architecture Pool Supervisor Milestone Assignment** (#7521): Added a "PR Workflow
-- 
2.52.0