chore(ci): capture nox output as CI artifacts and teach agents to read them
CI / lint (push) Has been cancelled
CI / security (push) Has been cancelled
CI / typecheck (push) Has been cancelled
CI / quality (push) Has been cancelled
CI / benchmark-publish (push) Has been cancelled
CI / status-check (push) Has been cancelled
CI / unit_tests (push) Has been cancelled
CI / integration_tests (push) Has been cancelled
CI / e2e_tests (push) Has been cancelled
CI / coverage (push) Has been cancelled
CI / benchmark-regression (push) Has been cancelled
CI / build (push) Has been cancelled
CI / docker (push) Has been cancelled
CI / helm (push) Has been cancelled
CI / lint (push) Has been cancelled
CI / security (push) Has been cancelled
CI / typecheck (push) Has been cancelled
CI / quality (push) Has been cancelled
CI / benchmark-publish (push) Has been cancelled
CI / status-check (push) Has been cancelled
CI / unit_tests (push) Has been cancelled
CI / integration_tests (push) Has been cancelled
CI / e2e_tests (push) Has been cancelled
CI / coverage (push) Has been cancelled
CI / benchmark-regression (push) Has been cancelled
CI / build (push) Has been cancelled
CI / docker (push) Has been cancelled
CI / helm (push) Has been cancelled
All 8 nox-running CI jobs in .forgejo/workflows/ci.yml now capture
stdout+stderr to build/nox-<job>-output.log via `2>&1 | tee` and upload
the log as a named Forgejo artifact (if: always(), retention-days: 30).
Artifact names follow the pattern ci-logs-<job>:
ci-logs-lint, ci-logs-typecheck, ci-logs-security, ci-logs-quality,
ci-logs-unit-tests, ci-logs-integration-tests, ci-logs-e2e-tests,
ci-logs-coverage
Seven agent definitions updated with a CI Log Artifacts section:
ca-pr-checker.md: artifact table + curl download instructions; Step 2
now downloads the relevant artifact before dispatching fix subagents.
ca-lint-fixer.md, ca-typecheck-fixer.md, ca-unit-test-runner.md,
ca-integration-test-runner.md, ca-coverage-checker.md,
ca-pr-self-reviewer.md: each receives a section explaining which
artifact corresponds to its domain and how to use it.
Design notes:
- tee (not redirect) preserves output in CI job logs AND captures to file
- if: always() ensures artifacts are available even when the job fails
- Multi-session jobs (lint, security) use tee -a to append to one file
- Existing coverage-reports artifact preserved alongside ci-logs-coverage
ISSUES CLOSED: #2750
This commit was merged in pull request #2782.
This commit is contained in:
+82
-11
@@ -37,16 +37,25 @@ jobs:
|
||||
|
||||
- name: Run lint via nox
|
||||
run: |
|
||||
nox -s lint
|
||||
mkdir -p build
|
||||
nox -s lint 2>&1 | tee build/nox-lint-output.log
|
||||
env:
|
||||
NOX_DEFAULT_VENV_BACKEND: uv
|
||||
|
||||
- name: Run format check via nox
|
||||
run: |
|
||||
nox -s format -- --check
|
||||
nox -s format -- --check 2>&1 | tee -a build/nox-lint-output.log
|
||||
env:
|
||||
NOX_DEFAULT_VENV_BACKEND: uv
|
||||
|
||||
- name: Upload lint log artifact
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: ci-logs-lint
|
||||
path: build/nox-lint-output.log
|
||||
retention-days: 30
|
||||
|
||||
typecheck:
|
||||
runs-on: docker
|
||||
container:
|
||||
@@ -72,10 +81,19 @@ jobs:
|
||||
|
||||
- name: Run typecheck via nox
|
||||
run: |
|
||||
nox -s typecheck
|
||||
mkdir -p build
|
||||
nox -s typecheck 2>&1 | tee build/nox-typecheck-output.log
|
||||
env:
|
||||
NOX_DEFAULT_VENV_BACKEND: uv
|
||||
|
||||
- name: Upload typecheck log artifact
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: ci-logs-typecheck
|
||||
path: build/nox-typecheck-output.log
|
||||
retention-days: 30
|
||||
|
||||
security:
|
||||
runs-on: docker
|
||||
container:
|
||||
@@ -101,16 +119,25 @@ jobs:
|
||||
|
||||
- name: Run security scan via nox
|
||||
run: |
|
||||
nox -s security_scan
|
||||
mkdir -p build
|
||||
nox -s security_scan 2>&1 | tee build/nox-security-output.log
|
||||
env:
|
||||
NOX_DEFAULT_VENV_BACKEND: uv
|
||||
|
||||
- name: Run dead code detection via nox
|
||||
run: |
|
||||
nox -s dead_code
|
||||
nox -s dead_code 2>&1 | tee -a build/nox-security-output.log
|
||||
env:
|
||||
NOX_DEFAULT_VENV_BACKEND: uv
|
||||
|
||||
- name: Upload security log artifact
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: ci-logs-security
|
||||
path: build/nox-security-output.log
|
||||
retention-days: 30
|
||||
|
||||
quality:
|
||||
runs-on: docker
|
||||
container:
|
||||
@@ -136,10 +163,19 @@ jobs:
|
||||
|
||||
- name: Run complexity check via nox
|
||||
run: |
|
||||
nox -s complexity
|
||||
mkdir -p build
|
||||
nox -s complexity 2>&1 | tee build/nox-quality-output.log
|
||||
env:
|
||||
NOX_DEFAULT_VENV_BACKEND: uv
|
||||
|
||||
- name: Upload quality log artifact
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: ci-logs-quality
|
||||
path: build/nox-quality-output.log
|
||||
retention-days: 30
|
||||
|
||||
unit_tests:
|
||||
runs-on: docker
|
||||
container:
|
||||
@@ -178,10 +214,19 @@ jobs:
|
||||
|
||||
- name: Run unit tests via nox
|
||||
run: |
|
||||
nox -s unit_tests
|
||||
mkdir -p build
|
||||
nox -s unit_tests 2>&1 | tee build/nox-unit-tests-output.log
|
||||
env:
|
||||
NOX_DEFAULT_VENV_BACKEND: uv
|
||||
|
||||
- name: Upload unit tests log artifact
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: ci-logs-unit-tests
|
||||
path: build/nox-unit-tests-output.log
|
||||
retention-days: 30
|
||||
|
||||
integration_tests:
|
||||
runs-on: docker
|
||||
container:
|
||||
@@ -220,7 +265,8 @@ jobs:
|
||||
|
||||
- name: Run integration tests via nox
|
||||
run: |
|
||||
nox -s integration_tests
|
||||
mkdir -p build
|
||||
nox -s integration_tests 2>&1 | tee build/nox-integration-tests-output.log
|
||||
env:
|
||||
NOX_DEFAULT_VENV_BACKEND: uv
|
||||
CLEVERAGENTS_REQUIRE_HELM_RENDER_ASSERTIONS: "true"
|
||||
@@ -231,6 +277,14 @@ jobs:
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
|
||||
- name: Upload integration tests log artifact
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: ci-logs-integration-tests
|
||||
path: build/nox-integration-tests-output.log
|
||||
retention-days: 30
|
||||
|
||||
e2e_tests:
|
||||
runs-on: docker
|
||||
timeout-minutes: 45
|
||||
@@ -257,13 +311,22 @@ jobs:
|
||||
|
||||
- name: Run E2E tests via nox
|
||||
run: |
|
||||
nox -s e2e_tests
|
||||
mkdir -p build
|
||||
nox -s e2e_tests 2>&1 | tee build/nox-e2e-tests-output.log
|
||||
env:
|
||||
NOX_DEFAULT_VENV_BACKEND: uv
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
GOOGLE_API_KEY: ${{ secrets.GOOGLE_API_KEY }}
|
||||
|
||||
- name: Upload E2E tests log artifact
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: ci-logs-e2e-tests
|
||||
path: build/nox-e2e-tests-output.log
|
||||
retention-days: 30
|
||||
|
||||
coverage:
|
||||
runs-on: docker
|
||||
container:
|
||||
@@ -292,9 +355,9 @@ jobs:
|
||||
id: coverage
|
||||
run: |
|
||||
mkdir -p build
|
||||
nox -s coverage_report 2>&1 | tee build/coverage-output.txt
|
||||
nox -s coverage_report 2>&1 | tee build/nox-coverage-output.log
|
||||
# Extract the single-line CI summary from nox output
|
||||
grep -E '^(nox > )?COVERAGE (OK|FAILED):' build/coverage-output.txt || true
|
||||
grep -E '^(nox > )?COVERAGE (OK|FAILED):' build/nox-coverage-output.log || true
|
||||
env:
|
||||
NOX_DEFAULT_VENV_BACKEND: uv
|
||||
|
||||
@@ -320,6 +383,14 @@ jobs:
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Upload coverage log artifact
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v3
|
||||
with:
|
||||
name: ci-logs-coverage
|
||||
path: build/nox-coverage-output.log
|
||||
retention-days: 30
|
||||
|
||||
- name: Upload coverage artifacts
|
||||
if: always()
|
||||
uses: actions/upload-artifact@v3
|
||||
|
||||
@@ -44,6 +44,21 @@ for coding standards and quality gates. Key rules for coverage:
|
||||
- All mocks must live under `features/mocks/` — never in production code.
|
||||
- Follow the BDD Test Organization Guidelines for test file structure.
|
||||
|
||||
## CI Log Artifacts
|
||||
|
||||
When invoked after a CI failure, you may be provided with the contents of
|
||||
the `ci-logs-coverage` artifact (log file: `build/nox-coverage-output.log`).
|
||||
This artifact contains the complete stdout/stderr output from the
|
||||
`coverage_report` nox session as it ran in CI, including the coverage
|
||||
percentage and any threshold failure messages.
|
||||
|
||||
**If artifact log content is provided:** Read it first to determine the
|
||||
current coverage percentage and which files are under-covered before running
|
||||
nox locally. This avoids a redundant nox run and gives you immediate context
|
||||
on what needs to be improved.
|
||||
|
||||
**If no artifact content is provided:** Proceed directly to Step 1 below.
|
||||
|
||||
## Process
|
||||
|
||||
### Step 1: Run Coverage Report
|
||||
|
||||
@@ -40,6 +40,20 @@ for coding standards, testing requirements, and quality gates. Key rules:
|
||||
services and real dependencies.
|
||||
- Test failures are **blocking** — resolve before proceeding.
|
||||
|
||||
## CI Log Artifacts
|
||||
|
||||
When invoked after a CI failure, you may be provided with the contents of
|
||||
the `ci-logs-integration-tests` artifact (log file:
|
||||
`build/nox-integration-tests-output.log`). This artifact contains the
|
||||
complete stdout/stderr output from the `integration_tests` nox session
|
||||
(Robot Framework) as it ran in CI.
|
||||
|
||||
**If artifact log content is provided:** Read it first to identify the
|
||||
failing Robot test suites, keywords, and error messages before running nox
|
||||
locally. This avoids a redundant nox run and gives you precise failure context.
|
||||
|
||||
**If no artifact content is provided:** Proceed directly to Step 1 below.
|
||||
|
||||
## Process
|
||||
|
||||
### Step 1: Run Integration Tests
|
||||
|
||||
@@ -40,6 +40,19 @@ for coding standards and quality gates. Key rules for lint fixing:
|
||||
- Source code in `src/cleveragents/`, tests in `features/` and `robot/`,
|
||||
mocks only in `features/mocks/`.
|
||||
|
||||
## CI Log Artifacts
|
||||
|
||||
When invoked after a CI failure, you may be provided with the contents of
|
||||
the `ci-logs-lint` artifact (log file: `build/nox-lint-output.log`). This
|
||||
artifact contains the complete stdout/stderr output from the `lint` and
|
||||
`format` nox sessions as they ran in CI.
|
||||
|
||||
**If artifact log content is provided:** Read it first to understand the
|
||||
exact errors before running nox locally. This avoids a redundant nox run
|
||||
and gives you precise line numbers and error codes immediately.
|
||||
|
||||
**If no artifact content is provided:** Proceed directly to Step 1 below.
|
||||
|
||||
## Process
|
||||
|
||||
### Step 1: Run Lint
|
||||
|
||||
@@ -83,6 +83,47 @@ You will be given:
|
||||
- **Forgejo PAT** — for HTTPS git auth (needed for standalone clone)
|
||||
- **Git full name / email** — for git identity (needed for standalone clone)
|
||||
|
||||
## CI Log Artifacts
|
||||
|
||||
Every nox-running CI job uploads its output as a Forgejo artifact. **Before
|
||||
dispatching any fix subagent, download and read the relevant artifact log.**
|
||||
This gives you the exact error output without needing to re-run nox locally.
|
||||
|
||||
### Artifact Names and Log Files
|
||||
|
||||
| CI Job | Artifact Name | Log File |
|
||||
|---|---|---|
|
||||
| `lint` | `ci-logs-lint` | `build/nox-lint-output.log` |
|
||||
| `typecheck` | `ci-logs-typecheck` | `build/nox-typecheck-output.log` |
|
||||
| `security` | `ci-logs-security` | `build/nox-security-output.log` |
|
||||
| `quality` | `ci-logs-quality` | `build/nox-quality-output.log` |
|
||||
| `unit_tests` | `ci-logs-unit-tests` | `build/nox-unit-tests-output.log` |
|
||||
| `integration_tests` | `ci-logs-integration-tests` | `build/nox-integration-tests-output.log` |
|
||||
| `e2e_tests` | `ci-logs-e2e-tests` | `build/nox-e2e-tests-output.log` |
|
||||
| `coverage` | `ci-logs-coverage` | `build/nox-coverage-output.log` |
|
||||
|
||||
### How to Download Artifacts
|
||||
|
||||
Use the Forgejo API to list and download artifacts for the workflow run
|
||||
associated with the failing PR commit:
|
||||
|
||||
```bash
|
||||
# List artifacts for the run (replace RUN_ID with the actual run ID)
|
||||
curl -s -H "Authorization: token <FORGEJO_PAT>" \
|
||||
"https://git.cleverthis.com/api/v1/repos/cleveragents/cleveragents-core/actions/runs/<RUN_ID>/artifacts"
|
||||
|
||||
# Download a specific artifact (replace ARTIFACT_ID)
|
||||
curl -L -H "Authorization: token <FORGEJO_PAT>" \
|
||||
"https://git.cleverthis.com/api/v1/repos/cleveragents/cleveragents-core/actions/artifacts/<ARTIFACT_ID>/zip" \
|
||||
-o /tmp/artifact.zip
|
||||
unzip -p /tmp/artifact.zip # pipe to stdout to read the log
|
||||
```
|
||||
|
||||
**Workflow:** Identify the failing check → download its artifact log → read
|
||||
the full error output → pass the relevant error context to the fix subagent.
|
||||
This avoids redundant local nox runs and gives the subagent precise failure
|
||||
information.
|
||||
|
||||
## Process
|
||||
|
||||
### Step 1: Wait for Checks
|
||||
@@ -91,9 +132,10 @@ complete.
|
||||
|
||||
### Step 2: If Any Checks Fail
|
||||
|
||||
1. **Identify the failure** from the check output/logs.
|
||||
1. **Download and read the CI log artifact** for the failing job (see CI Log
|
||||
Artifacts section above). Extract the relevant error messages.
|
||||
2. **Fix the issue** in the working directory. Depending on the failure type,
|
||||
invoke the appropriate subagent:
|
||||
invoke the appropriate subagent with the artifact log content as context:
|
||||
- Lint failure → `ca-lint-fixer`
|
||||
- Type check failure → `ca-typecheck-fixer`
|
||||
- Unit test failure → `ca-unit-test-runner`
|
||||
|
||||
@@ -41,6 +41,31 @@ You receive:
|
||||
- **workdir**: working directory (optional, defaults to `/app`)
|
||||
- **spec_context**: specification context or module names to review against
|
||||
|
||||
## CI Log Artifacts
|
||||
|
||||
Every nox-running CI job uploads its output as a Forgejo artifact. When
|
||||
reviewing a PR, you can download these artifacts to read the full CI output
|
||||
without re-running nox locally. This is especially useful when CI is failing
|
||||
and you need to understand the exact errors before invoking `ca-pr-checker`.
|
||||
|
||||
### Artifact Names
|
||||
|
||||
| CI Job | Artifact Name | Log File |
|
||||
|---|---|---|
|
||||
| `lint` | `ci-logs-lint` | `build/nox-lint-output.log` |
|
||||
| `typecheck` | `ci-logs-typecheck` | `build/nox-typecheck-output.log` |
|
||||
| `security` | `ci-logs-security` | `build/nox-security-output.log` |
|
||||
| `quality` | `ci-logs-quality` | `build/nox-quality-output.log` |
|
||||
| `unit_tests` | `ci-logs-unit-tests` | `build/nox-unit-tests-output.log` |
|
||||
| `integration_tests` | `ci-logs-integration-tests` | `build/nox-integration-tests-output.log` |
|
||||
| `e2e_tests` | `ci-logs-e2e-tests` | `build/nox-e2e-tests-output.log` |
|
||||
| `coverage` | `ci-logs-coverage` | `build/nox-coverage-output.log` |
|
||||
|
||||
Use the Forgejo API to list artifacts for the workflow run associated with
|
||||
the PR's head commit, then download the relevant artifact zip to read the
|
||||
log content. Pass the artifact log content to `ca-pr-checker` when invoking
|
||||
it to fix CI failures — this gives the fixer precise error context.
|
||||
|
||||
## Required Reading
|
||||
|
||||
Before beginning any review, you must be operating with knowledge of:
|
||||
|
||||
@@ -40,6 +40,19 @@ for coding standards and quality gates. Key rules for type checking:
|
||||
- **Never modify Pyright configuration** to disable checks.
|
||||
- Type checking must pass via `nox -s typecheck`.
|
||||
|
||||
## CI Log Artifacts
|
||||
|
||||
When invoked after a CI failure, you may be provided with the contents of
|
||||
the `ci-logs-typecheck` artifact (log file: `build/nox-typecheck-output.log`).
|
||||
This artifact contains the complete stdout/stderr output from the `typecheck`
|
||||
nox session as it ran in CI.
|
||||
|
||||
**If artifact log content is provided:** Read it first to understand the
|
||||
exact Pyright errors before running nox locally. This avoids a redundant nox
|
||||
run and gives you precise file paths, line numbers, and error codes immediately.
|
||||
|
||||
**If no artifact content is provided:** Proceed directly to Step 1 below.
|
||||
|
||||
## Process
|
||||
|
||||
### Step 1: Run Typecheck
|
||||
|
||||
@@ -41,6 +41,19 @@ for coding standards, testing requirements, and quality gates. Key rules:
|
||||
- Follow the **BDD Test Organization Guidelines** for step file structure.
|
||||
- Follow the **TDD Issue Test Tags** system for bug-related tests.
|
||||
|
||||
## CI Log Artifacts
|
||||
|
||||
When invoked after a CI failure, you may be provided with the contents of
|
||||
the `ci-logs-unit-tests` artifact (log file: `build/nox-unit-tests-output.log`).
|
||||
This artifact contains the complete stdout/stderr output from the `unit_tests`
|
||||
nox session (Behave) as it ran in CI.
|
||||
|
||||
**If artifact log content is provided:** Read it first to identify the
|
||||
failing scenarios, step definitions, and error messages before running nox
|
||||
locally. This avoids a redundant nox run and gives you precise failure context.
|
||||
|
||||
**If no artifact content is provided:** Proceed directly to Step 1 below.
|
||||
|
||||
## Process
|
||||
|
||||
### Step 1: Run Unit Tests
|
||||
|
||||
Reference in New Issue
Block a user