cleveragents/cleveragents-core
4
0
Fork 3
Code Issues 6.0k Pull Requests 115 Actions Packages Projects Releases Wiki Activity

perf(ci): reduce CI quality check execution time by parallelizing and caching #10845

Merged
HAL9000 merged 7 commits from test/v3.8.0-ci-quality-execution-time into master 2026-06-18 08:20:49 +00:00
Conversation 53 Commits 7 Files Changed 4
+74 -20
4 changed files with 74 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.forgejo/workflows/ci.yml
+67 -8
View File
1
 @@ -77,10 +77,18 @@ jobs:
uses: actions/cache@v3
with:
path: ~/.cache/uv
key: uv-${{ hashFiles('pyproject.toml') }}
key: uv-${{ hashFiles('uv.lock', 'pyproject.toml') }}
restore-keys: |
uv-
- name: Cache nox virtualenvs
uses: actions/cache@v3
with:
path: .nox
key: nox-lint-${{ env.PYTHON_VERSION }}-${{ hashFiles('uv.lock', 'pyproject.toml') }}
restore-keys: |
nox-lint-${{ env.PYTHON_VERSION }}-
- name: Run lint and format checks via nox
run: |
mkdir -p build
@@ -124,10 +132,18 @@ jobs:
uses: actions/cache@v3
with:
path: ~/.cache/uv
HAL9001 commented 2026-05-06 08:07:07 +00:00
Outdated
Review
Copy Link
Copy Source

BLOCKER: The unit_tests dependency was deliberately included in the original coverage job needs to prevent misleading coverage results when unit tests are failing. The original comment stated: unit_tests is included so coverage only runs after tests pass, preventing misleading results when tests are still in-flight or failing.

Removing lint, typecheck, security, and quality from coverage's needs is sound (those are independent static-analysis jobs). However, removing unit_tests is a correctness regression — coverage can now run and report passing metrics on code that has known failing tests.

Recommended fix: Add needs: [unit_tests] back to the coverage job:

        # Coverage runs the full unit-test suite independently under slipcover.
        # It does not wait for lint/typecheck/security/quality (static-analysis
        # jobs that do not affect test results), but does wait for unit_tests
        # to avoid reporting coverage on code with known failing tests.
        needs: [unit_tests]

This preserves all parallelisation benefits against static-analysis jobs while retaining the safety gate against failing tests.

BLOCKER: The `unit_tests` dependency was deliberately included in the original `coverage` job `needs` to prevent misleading coverage results when unit tests are failing. The original comment stated: `unit_tests is included so coverage only runs after tests pass, preventing misleading results when tests are still in-flight or failing.` Removing `lint`, `typecheck`, `security`, and `quality` from coverage's `needs` is sound (those are independent static-analysis jobs). However, removing `unit_tests` is a correctness regression — coverage can now run and report passing metrics on code that has known failing tests. Recommended fix: Add `needs: [unit_tests]` back to the coverage job: ```yaml # Coverage runs the full unit-test suite independently under slipcover. # It does not wait for lint/typecheck/security/quality (static-analysis # jobs that do not affect test results), but does wait for unit_tests # to avoid reporting coverage on code with known failing tests. needs: [unit_tests] ``` This preserves all parallelisation benefits against static-analysis jobs while retaining the safety gate against failing tests.
key: uv-${{ hashFiles('pyproject.toml') }}
key: uv-${{ hashFiles('uv.lock', 'pyproject.toml') }}
restore-keys: |
uv-
- name: Cache nox virtualenvs
uses: actions/cache@v3
with:
path: .nox
key: nox-typecheck-${{ env.PYTHON_VERSION }}-${{ hashFiles('uv.lock', 'pyproject.toml') }}
restore-keys: |
nox-typecheck-${{ env.PYTHON_VERSION }}-
- name: Run typecheck via nox
run: |
mkdir -p build
@@ -170,10 +186,18 @@ jobs:
uses: actions/cache@v3
with:
path: ~/.cache/uv
key: uv-${{ hashFiles('pyproject.toml') }}
key: uv-${{ hashFiles('uv.lock', 'pyproject.toml') }}
restore-keys: |
uv-
- name: Cache nox virtualenvs
uses: actions/cache@v3
with:
path: .nox
key: nox-security-${{ env.PYTHON_VERSION }}-${{ hashFiles('uv.lock', 'pyproject.toml') }}
restore-keys: |
nox-security-${{ env.PYTHON_VERSION }}-
- name: Run security scan and dead-code detection via nox
run: |
mkdir -p build
1
 @@ -217,10 +241,18 @@ jobs:
uses: actions/cache@v3
with:
path: ~/.cache/uv
key: uv-${{ hashFiles('pyproject.toml') }}
key: uv-${{ hashFiles('uv.lock', 'pyproject.toml') }}
restore-keys: |
uv-
- name: Cache nox virtualenvs
uses: actions/cache@v3
with:
path: .nox
key: nox-quality-${{ env.PYTHON_VERSION }}-${{ hashFiles('uv.lock', 'pyproject.toml') }}
restore-keys: |
nox-quality-${{ env.PYTHON_VERSION }}-
- name: Run complexity check via nox
run: |
mkdir -p build
@@ -288,10 +320,18 @@ jobs:
uses: actions/cache@v3
with:
path: ~/.cache/uv
key: uv-${{ hashFiles('pyproject.toml') }}
key: uv-${{ hashFiles('uv.lock', 'pyproject.toml') }}
restore-keys: |
uv-
- name: Cache nox virtualenvs
uses: actions/cache@v3
with:
path: .nox
key: nox-unit-tests-${{ env.PYTHON_VERSION }}-${{ hashFiles('uv.lock', 'pyproject.toml') }}
restore-keys: |
nox-unit-tests-${{ env.PYTHON_VERSION }}-
- name: Run unit tests via nox
run: |
mkdir -p build
@@ -364,10 +404,18 @@ jobs:
uses: actions/cache@v3
with:
path: ~/.cache/uv
key: uv-${{ hashFiles('pyproject.toml') }}
key: uv-${{ hashFiles('uv.lock', 'pyproject.toml') }}
restore-keys: |
uv-
- name: Cache nox virtualenvs
uses: actions/cache@v3
with:
path: .nox
key: nox-integration-tests-${{ env.PYTHON_VERSION }}-${{ hashFiles('uv.lock', 'pyproject.toml') }}
restore-keys: |
nox-integration-tests-${{ env.PYTHON_VERSION }}-
- name: Run integration tests via nox
run: |
mkdir -p build
@@ -517,10 +565,18 @@ jobs:
uses: actions/cache@v3
with:
path: ~/.cache/uv
key: uv-${{ hashFiles('pyproject.toml') }}
key: uv-${{ hashFiles('uv.lock', 'pyproject.toml') }}
restore-keys: |
uv-
- name: Cache nox virtualenvs
uses: actions/cache@v3
with:
path: .nox
key: nox-build-${{ env.PYTHON_VERSION }}-${{ hashFiles('uv.lock', 'pyproject.toml') }}
restore-keys: |
nox-build-${{ env.PYTHON_VERSION }}-
- name: Build wheel via nox
run: |
mkdir -p build
@@ -544,10 +600,13 @@ jobs:
retention-days: 30
docker:
# Only gate on unit_tests -- the most critical functional check.
# lint/typecheck/security/quality run in parallel and are not
# prerequisites for a successful Docker image build.
# continue-on-error: true allows the status-check gate to pass even
# when the docker:dind runner is unavailable (infrastructure issue).
# The docker job still runs; only infrastructure failures are tolerated.
needs: [load-versions, lint, typecheck, security, quality, unit_tests]
needs: [unit_tests]
continue-on-error: true
runs-on: docker
container:
features/ci_workflow_validation.feature
+2 -3
View File
@@ -54,11 +54,10 @@ Feature: CI workflow validation
When I parse the CI workflow YAML
Then the workflow env should set "PYTHON_VERSION" to "3.13"
Scenario: CI workflow coverage job depends on lint and typecheck
Scenario: CI workflow coverage job depends on unit_tests
Given the CI workflow file at ".forgejo/workflows/ci.yml"
When I parse the CI workflow YAML
Then the job "coverage" should depend on "lint"
And the job "coverage" should depend on "typecheck"
Then the job "coverage" should depend on "unit_tests"
Scenario: All required nox sessions are referenced
Given the CI workflow file at ".forgejo/workflows/ci.yml"
features/coverage_threshold_enforcement.feature
+2 -2
View File
@@ -44,10 +44,10 @@ Feature: Coverage threshold enforcement
When I parse the coverage job from the CI workflow
Then the coverage job should run nox -s coverage_report
Scenario: CI workflow coverage job depends on lint and typecheck
Scenario: CI workflow coverage job depends on unit_tests
Given the CI workflow file exists
When I parse the coverage job from the CI workflow
Then the coverage job should depend on lint and typecheck
Then the coverage job should depend on unit_tests
Scenario: Coverage report session emits CI-parseable summary on success
Given the noxfile.py exists
features/steps/coverage_threshold_enforcement_steps.py
+3 -7
View File
@@ -214,17 +214,13 @@ def step_check_ci_coverage_nox(context: Any) -> None:
)
@then("the coverage job should depend on lint and typecheck")
@then("the coverage job should depend on unit_tests")
def step_check_ci_coverage_deps(context: Any) -> None:
job = context.ci_coverage_job
needs = job.get("needs", [])
if "lint" not in needs:
if "unit_tests" not in needs:
raise AssertionError(
f"CI coverage job does not depend on 'lint': needs={needs}"
)
if "typecheck" not in needs:
raise AssertionError(
f"CI coverage job does not depend on 'typecheck': needs={needs}"
f"CI coverage job does not depend on 'unit_tests': needs={needs}"
)