chore(ci): capture nox output as CI artifacts and teach agents to read them
CI / lint (push) Has been cancelled
CI / security (push) Has been cancelled
CI / typecheck (push) Has been cancelled
CI / quality (push) Has been cancelled
CI / benchmark-publish (push) Has been cancelled
CI / status-check (push) Has been cancelled
CI / unit_tests (push) Has been cancelled
CI / integration_tests (push) Has been cancelled
CI / e2e_tests (push) Has been cancelled
CI / coverage (push) Has been cancelled
CI / benchmark-regression (push) Has been cancelled
CI / build (push) Has been cancelled
CI / docker (push) Has been cancelled
CI / helm (push) Has been cancelled

All 8 nox-running CI jobs in .forgejo/workflows/ci.yml now capture
stdout+stderr to build/nox-<job>-output.log via `2>&1 | tee` and upload
the log as a named Forgejo artifact (if: always(), retention-days: 30).
Artifact names follow the pattern ci-logs-<job>:
  ci-logs-lint, ci-logs-typecheck, ci-logs-security, ci-logs-quality,
  ci-logs-unit-tests, ci-logs-integration-tests, ci-logs-e2e-tests,
  ci-logs-coverage

Seven agent definitions updated with a CI Log Artifacts section:
  ca-pr-checker.md: artifact table + curl download instructions; Step 2
    now downloads the relevant artifact before dispatching fix subagents.
  ca-lint-fixer.md, ca-typecheck-fixer.md, ca-unit-test-runner.md,
  ca-integration-test-runner.md, ca-coverage-checker.md,
  ca-pr-self-reviewer.md: each receives a section explaining which
    artifact corresponds to its domain and how to use it.

Design notes:
- tee (not redirect) preserves output in CI job logs AND captures to file
- if: always() ensures artifacts are available even when the job fails
- Multi-session jobs (lint, security) use tee -a to append to one file
- Existing coverage-reports artifact preserved alongside ci-logs-coverage

ISSUES CLOSED: #2750
This commit was merged in pull request #2782.
This commit is contained in:
2026-04-04 19:23:28 +00:00
committed by Forgejo
parent 03334aaa3d
commit 72e0db2592
8 changed files with 219 additions and 13 deletions
+82 -11
View File
@@ -37,16 +37,25 @@ jobs:
- name: Run lint via nox
run: |
nox -s lint
mkdir -p build
nox -s lint 2>&1 | tee build/nox-lint-output.log
env:
NOX_DEFAULT_VENV_BACKEND: uv
- name: Run format check via nox
run: |
nox -s format -- --check
nox -s format -- --check 2>&1 | tee -a build/nox-lint-output.log
env:
NOX_DEFAULT_VENV_BACKEND: uv
- name: Upload lint log artifact
if: always()
uses: actions/upload-artifact@v3
with:
name: ci-logs-lint
path: build/nox-lint-output.log
retention-days: 30
typecheck:
runs-on: docker
container:
@@ -72,10 +81,19 @@ jobs:
- name: Run typecheck via nox
run: |
nox -s typecheck
mkdir -p build
nox -s typecheck 2>&1 | tee build/nox-typecheck-output.log
env:
NOX_DEFAULT_VENV_BACKEND: uv
- name: Upload typecheck log artifact
if: always()
uses: actions/upload-artifact@v3
with:
name: ci-logs-typecheck
path: build/nox-typecheck-output.log
retention-days: 30
security:
runs-on: docker
container:
@@ -101,16 +119,25 @@ jobs:
- name: Run security scan via nox
run: |
nox -s security_scan
mkdir -p build
nox -s security_scan 2>&1 | tee build/nox-security-output.log
env:
NOX_DEFAULT_VENV_BACKEND: uv
- name: Run dead code detection via nox
run: |
nox -s dead_code
nox -s dead_code 2>&1 | tee -a build/nox-security-output.log
env:
NOX_DEFAULT_VENV_BACKEND: uv
- name: Upload security log artifact
if: always()
uses: actions/upload-artifact@v3
with:
name: ci-logs-security
path: build/nox-security-output.log
retention-days: 30
quality:
runs-on: docker
container:
@@ -136,10 +163,19 @@ jobs:
- name: Run complexity check via nox
run: |
nox -s complexity
mkdir -p build
nox -s complexity 2>&1 | tee build/nox-quality-output.log
env:
NOX_DEFAULT_VENV_BACKEND: uv
- name: Upload quality log artifact
if: always()
uses: actions/upload-artifact@v3
with:
name: ci-logs-quality
path: build/nox-quality-output.log
retention-days: 30
unit_tests:
runs-on: docker
container:
@@ -178,10 +214,19 @@ jobs:
- name: Run unit tests via nox
run: |
nox -s unit_tests
mkdir -p build
nox -s unit_tests 2>&1 | tee build/nox-unit-tests-output.log
env:
NOX_DEFAULT_VENV_BACKEND: uv
- name: Upload unit tests log artifact
if: always()
uses: actions/upload-artifact@v3
with:
name: ci-logs-unit-tests
path: build/nox-unit-tests-output.log
retention-days: 30
integration_tests:
runs-on: docker
container:
@@ -220,7 +265,8 @@ jobs:
- name: Run integration tests via nox
run: |
nox -s integration_tests
mkdir -p build
nox -s integration_tests 2>&1 | tee build/nox-integration-tests-output.log
env:
NOX_DEFAULT_VENV_BACKEND: uv
CLEVERAGENTS_REQUIRE_HELM_RENDER_ASSERTIONS: "true"
@@ -231,6 +277,14 @@ jobs:
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
- name: Upload integration tests log artifact
if: always()
uses: actions/upload-artifact@v3
with:
name: ci-logs-integration-tests
path: build/nox-integration-tests-output.log
retention-days: 30
e2e_tests:
runs-on: docker
timeout-minutes: 45
@@ -257,13 +311,22 @@ jobs:
- name: Run E2E tests via nox
run: |
nox -s e2e_tests
mkdir -p build
nox -s e2e_tests 2>&1 | tee build/nox-e2e-tests-output.log
env:
NOX_DEFAULT_VENV_BACKEND: uv
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
- name: Upload E2E tests log artifact
if: always()
uses: actions/upload-artifact@v3
with:
name: ci-logs-e2e-tests
path: build/nox-e2e-tests-output.log
retention-days: 30
coverage:
runs-on: docker
container:
@@ -292,9 +355,9 @@ jobs:
id: coverage
run: |
mkdir -p build
nox -s coverage_report 2>&1 | tee build/coverage-output.txt
nox -s coverage_report 2>&1 | tee build/nox-coverage-output.log
# Extract the single-line CI summary from nox output
grep -E '^(nox > )?COVERAGE (OK|FAILED):' build/coverage-output.txt || true
grep -E '^(nox > )?COVERAGE (OK|FAILED):' build/nox-coverage-output.log || true
env:
NOX_DEFAULT_VENV_BACKEND: uv
@@ -320,6 +383,14 @@ jobs:
exit 1
fi
- name: Upload coverage log artifact
if: always()
uses: actions/upload-artifact@v3
with:
name: ci-logs-coverage
path: build/nox-coverage-output.log
retention-days: 30
- name: Upload coverage artifacts
if: always()
uses: actions/upload-artifact@v3
+15
View File
@@ -44,6 +44,21 @@ for coding standards and quality gates. Key rules for coverage:
- All mocks must live under `features/mocks/` — never in production code.
- Follow the BDD Test Organization Guidelines for test file structure.
## CI Log Artifacts
When invoked after a CI failure, you may be provided with the contents of
the `ci-logs-coverage` artifact (log file: `build/nox-coverage-output.log`).
This artifact contains the complete stdout/stderr output from the
`coverage_report` nox session as it ran in CI, including the coverage
percentage and any threshold failure messages.
**If artifact log content is provided:** Read it first to determine the
current coverage percentage and which files are under-covered before running
nox locally. This avoids a redundant nox run and gives you immediate context
on what needs to be improved.
**If no artifact content is provided:** Proceed directly to Step 1 below.
## Process
### Step 1: Run Coverage Report
@@ -40,6 +40,20 @@ for coding standards, testing requirements, and quality gates. Key rules:
services and real dependencies.
- Test failures are **blocking** — resolve before proceeding.
## CI Log Artifacts
When invoked after a CI failure, you may be provided with the contents of
the `ci-logs-integration-tests` artifact (log file:
`build/nox-integration-tests-output.log`). This artifact contains the
complete stdout/stderr output from the `integration_tests` nox session
(Robot Framework) as it ran in CI.
**If artifact log content is provided:** Read it first to identify the
failing Robot test suites, keywords, and error messages before running nox
locally. This avoids a redundant nox run and gives you precise failure context.
**If no artifact content is provided:** Proceed directly to Step 1 below.
## Process
### Step 1: Run Integration Tests
+13
View File
@@ -40,6 +40,19 @@ for coding standards and quality gates. Key rules for lint fixing:
- Source code in `src/cleveragents/`, tests in `features/` and `robot/`,
mocks only in `features/mocks/`.
## CI Log Artifacts
When invoked after a CI failure, you may be provided with the contents of
the `ci-logs-lint` artifact (log file: `build/nox-lint-output.log`). This
artifact contains the complete stdout/stderr output from the `lint` and
`format` nox sessions as they ran in CI.
**If artifact log content is provided:** Read it first to understand the
exact errors before running nox locally. This avoids a redundant nox run
and gives you precise line numbers and error codes immediately.
**If no artifact content is provided:** Proceed directly to Step 1 below.
## Process
### Step 1: Run Lint
+44 -2
View File
@@ -83,6 +83,47 @@ You will be given:
- **Forgejo PAT** — for HTTPS git auth (needed for standalone clone)
- **Git full name / email** — for git identity (needed for standalone clone)
## CI Log Artifacts
Every nox-running CI job uploads its output as a Forgejo artifact. **Before
dispatching any fix subagent, download and read the relevant artifact log.**
This gives you the exact error output without needing to re-run nox locally.
### Artifact Names and Log Files
| CI Job | Artifact Name | Log File |
|---|---|---|
| `lint` | `ci-logs-lint` | `build/nox-lint-output.log` |
| `typecheck` | `ci-logs-typecheck` | `build/nox-typecheck-output.log` |
| `security` | `ci-logs-security` | `build/nox-security-output.log` |
| `quality` | `ci-logs-quality` | `build/nox-quality-output.log` |
| `unit_tests` | `ci-logs-unit-tests` | `build/nox-unit-tests-output.log` |
| `integration_tests` | `ci-logs-integration-tests` | `build/nox-integration-tests-output.log` |
| `e2e_tests` | `ci-logs-e2e-tests` | `build/nox-e2e-tests-output.log` |
| `coverage` | `ci-logs-coverage` | `build/nox-coverage-output.log` |
### How to Download Artifacts
Use the Forgejo API to list and download artifacts for the workflow run
associated with the failing PR commit:
```bash
# List artifacts for the run (replace RUN_ID with the actual run ID)
curl -s -H "Authorization: token <FORGEJO_PAT>" \
"https://git.cleverthis.com/api/v1/repos/cleveragents/cleveragents-core/actions/runs/<RUN_ID>/artifacts"
# Download a specific artifact (replace ARTIFACT_ID)
curl -L -H "Authorization: token <FORGEJO_PAT>" \
"https://git.cleverthis.com/api/v1/repos/cleveragents/cleveragents-core/actions/artifacts/<ARTIFACT_ID>/zip" \
-o /tmp/artifact.zip
unzip -p /tmp/artifact.zip # pipe to stdout to read the log
```
**Workflow:** Identify the failing check → download its artifact log → read
the full error output → pass the relevant error context to the fix subagent.
This avoids redundant local nox runs and gives the subagent precise failure
information.
## Process
### Step 1: Wait for Checks
@@ -91,9 +132,10 @@ complete.
### Step 2: If Any Checks Fail
1. **Identify the failure** from the check output/logs.
1. **Download and read the CI log artifact** for the failing job (see CI Log
Artifacts section above). Extract the relevant error messages.
2. **Fix the issue** in the working directory. Depending on the failure type,
invoke the appropriate subagent:
invoke the appropriate subagent with the artifact log content as context:
- Lint failure → `ca-lint-fixer`
- Type check failure → `ca-typecheck-fixer`
- Unit test failure → `ca-unit-test-runner`
+25
View File
@@ -41,6 +41,31 @@ You receive:
- **workdir**: working directory (optional, defaults to `/app`)
- **spec_context**: specification context or module names to review against
## CI Log Artifacts
Every nox-running CI job uploads its output as a Forgejo artifact. When
reviewing a PR, you can download these artifacts to read the full CI output
without re-running nox locally. This is especially useful when CI is failing
and you need to understand the exact errors before invoking `ca-pr-checker`.
### Artifact Names
| CI Job | Artifact Name | Log File |
|---|---|---|
| `lint` | `ci-logs-lint` | `build/nox-lint-output.log` |
| `typecheck` | `ci-logs-typecheck` | `build/nox-typecheck-output.log` |
| `security` | `ci-logs-security` | `build/nox-security-output.log` |
| `quality` | `ci-logs-quality` | `build/nox-quality-output.log` |
| `unit_tests` | `ci-logs-unit-tests` | `build/nox-unit-tests-output.log` |
| `integration_tests` | `ci-logs-integration-tests` | `build/nox-integration-tests-output.log` |
| `e2e_tests` | `ci-logs-e2e-tests` | `build/nox-e2e-tests-output.log` |
| `coverage` | `ci-logs-coverage` | `build/nox-coverage-output.log` |
Use the Forgejo API to list artifacts for the workflow run associated with
the PR's head commit, then download the relevant artifact zip to read the
log content. Pass the artifact log content to `ca-pr-checker` when invoking
it to fix CI failures — this gives the fixer precise error context.
## Required Reading
Before beginning any review, you must be operating with knowledge of:
+13
View File
@@ -40,6 +40,19 @@ for coding standards and quality gates. Key rules for type checking:
- **Never modify Pyright configuration** to disable checks.
- Type checking must pass via `nox -s typecheck`.
## CI Log Artifacts
When invoked after a CI failure, you may be provided with the contents of
the `ci-logs-typecheck` artifact (log file: `build/nox-typecheck-output.log`).
This artifact contains the complete stdout/stderr output from the `typecheck`
nox session as it ran in CI.
**If artifact log content is provided:** Read it first to understand the
exact Pyright errors before running nox locally. This avoids a redundant nox
run and gives you precise file paths, line numbers, and error codes immediately.
**If no artifact content is provided:** Proceed directly to Step 1 below.
## Process
### Step 1: Run Typecheck
+13
View File
@@ -41,6 +41,19 @@ for coding standards, testing requirements, and quality gates. Key rules:
- Follow the **BDD Test Organization Guidelines** for step file structure.
- Follow the **TDD Issue Test Tags** system for bug-related tests.
## CI Log Artifacts
When invoked after a CI failure, you may be provided with the contents of
the `ci-logs-unit-tests` artifact (log file: `build/nox-unit-tests-output.log`).
This artifact contains the complete stdout/stderr output from the `unit_tests`
nox session (Behave) as it ran in CI.
**If artifact log content is provided:** Read it first to identify the
failing scenarios, step definitions, and error messages before running nox
locally. This avoids a redundant nox run and gives you precise failure context.
**If no artifact content is provided:** Proceed directly to Step 1 below.
## Process
### Step 1: Run Unit Tests