cleveragents/cleveragents-core

Fork 3

TEST-INFRA: [ci-execution-time] Split unit_tests job for parallel execution #6889

New issue

Open

opened 2026-04-10 04:50:40 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-10 04:50:40 +00:00

Owner

Background

The unit_tests CI job is the longest-running job in the pipeline, creating a significant bottleneck in the feedback loop for developers. Analysis of recent CI logs shows:

ci_logs_pr6723_unit_tests.log: ~3 minutes 36 seconds
ci_logs_unit_tests_pull_request.log: ~5 minutes 19 seconds

In contrast, the lint job completes in under 30 seconds. The unit_tests job is a major bottleneck in the CI/CD pipeline and slows developer iteration velocity.

Current Behaviour

The unit_tests CI job runs all Behave BDD scenarios sequentially in a single job, taking up to 5 minutes 19 seconds to complete. This is the longest-running required check and delays merge feedback.

Expected Behaviour

The unit_tests job is split into multiple parallel jobs (e.g., unit_tests_1, unit_tests_2, unit_tests_3, unit_tests_4), each running a distinct, non-overlapping subset of the test suite. The total wall-clock time for unit test feedback is reduced by ≥50%.

Acceptance Criteria

The test suite is divided into logical groups (by feature, component, or file distribution)
Parallel jobs run concurrently in Forgejo Actions
Coverage aggregation across parallel jobs produces correct, combined results
All parallel unit test jobs are required checks for merge
No test scenario is skipped or duplicated across jobs
Total unit test CI time is reduced by ≥50% compared to the current baseline

Supporting Information

Duplicate check: Searched for CI, execution time, parallelize, split tests — 0 existing issues found
Relevant CI file: .forgejo/workflows/ci.yml — unit_tests job definition
Nox session: nox -s unit_tests (Behave BDD runner)
Framework: Behave (Gherkin/BDD) — test splitting must respect feature file boundaries

Metadata

Branch: ci/split-unit-tests-parallel-execution
Commit Message: ci(pipeline): split unit_tests job for parallel execution
Milestone: (none — see backlog note below)
Parent Epic: (to be linked — see orphan note)

Subtasks

Audit the full Behave test suite to identify logical groupings suitable for parallel execution
Design a test-splitting strategy (by feature directory, tag, or file distribution) that avoids duplication and gaps
Update .forgejo/workflows/ci.yml to define parallel unit_tests_1, unit_tests_2, unit_tests_3, unit_tests_4 jobs
Configure coverage artifact upload per parallel job and add a coverage-merge step
Ensure all parallel unit_tests_* jobs are listed as required checks for merge
Validate that the combined coverage report still meets the ≥97% threshold
Update CONTRIBUTING.md if the CI job names or invocation instructions change

Definition of Done

unit_tests job is replaced by parallel unit_tests_* jobs in ci.yml
Each parallel job runs a distinct, non-overlapping subset of Behave scenarios
Total wall-clock time for unit test CI feedback is reduced by ≥50% vs. baseline
Coverage aggregation across parallel jobs produces a single correct combined report
All parallel unit test jobs are required checks for merge (status-check gate updated)
No Behave scenario is skipped or duplicated across parallel jobs
All nox stages pass
Coverage >= 97%

Backlog note: This issue was discovered during autonomous operation
on milestone (unspecified). It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: new-issue-creator

## Background The `unit_tests` CI job is the longest-running job in the pipeline, creating a significant bottleneck in the feedback loop for developers. Analysis of recent CI logs shows: - **`ci_logs_pr6723_unit_tests.log`**: ~3 minutes 36 seconds - **`ci_logs_unit_tests_pull_request.log`**: ~5 minutes 19 seconds In contrast, the `lint` job completes in under 30 seconds. The `unit_tests` job is a major bottleneck in the CI/CD pipeline and slows developer iteration velocity. ## Current Behaviour The `unit_tests` CI job runs all Behave BDD scenarios sequentially in a single job, taking up to 5 minutes 19 seconds to complete. This is the longest-running required check and delays merge feedback. ## Expected Behaviour The `unit_tests` job is split into multiple parallel jobs (e.g., `unit_tests_1`, `unit_tests_2`, `unit_tests_3`, `unit_tests_4`), each running a distinct, non-overlapping subset of the test suite. The total wall-clock time for unit test feedback is reduced by ≥50%. ## Acceptance Criteria - The test suite is divided into logical groups (by feature, component, or file distribution) - Parallel jobs run concurrently in Forgejo Actions - Coverage aggregation across parallel jobs produces correct, combined results - All parallel unit test jobs are required checks for merge - No test scenario is skipped or duplicated across jobs - Total unit test CI time is reduced by ≥50% compared to the current baseline ## Supporting Information - **Duplicate check**: Searched for `CI`, `execution time`, `parallelize`, `split tests` — 0 existing issues found - **Relevant CI file**: `.forgejo/workflows/ci.yml` — `unit_tests` job definition - **Nox session**: `nox -s unit_tests` (Behave BDD runner) - **Framework**: Behave (Gherkin/BDD) — test splitting must respect feature file boundaries --- ## Metadata - **Branch**: `ci/split-unit-tests-parallel-execution` - **Commit Message**: `ci(pipeline): split unit_tests job for parallel execution` - **Milestone**: *(none — see backlog note below)* - **Parent Epic**: *(to be linked — see orphan note)* ## Subtasks - [ ] Audit the full Behave test suite to identify logical groupings suitable for parallel execution - [ ] Design a test-splitting strategy (by feature directory, tag, or file distribution) that avoids duplication and gaps - [ ] Update `.forgejo/workflows/ci.yml` to define parallel `unit_tests_1`, `unit_tests_2`, `unit_tests_3`, `unit_tests_4` jobs - [ ] Configure coverage artifact upload per parallel job and add a coverage-merge step - [ ] Ensure all parallel `unit_tests_*` jobs are listed as required checks for merge - [ ] Validate that the combined coverage report still meets the ≥97% threshold - [ ] Update `CONTRIBUTING.md` if the CI job names or invocation instructions change ## Definition of Done - [ ] `unit_tests` job is replaced by parallel `unit_tests_*` jobs in `ci.yml` - [ ] Each parallel job runs a distinct, non-overlapping subset of Behave scenarios - [ ] Total wall-clock time for unit test CI feedback is reduced by ≥50% vs. baseline - [ ] Coverage aggregation across parallel jobs produces a single correct combined report - [ ] All parallel unit test jobs are required checks for merge (status-check gate updated) - [ ] No Behave scenario is skipped or duplicated across parallel jobs - [ ] All nox stages pass - [ ] Coverage >= 97% --- > **Backlog note:** This issue was discovered during autonomous operation > on milestone *(unspecified)*. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: new-issue-creator

HAL9000 added the

labels

2026-04-10 04:53:13 +00:00

HAL9000 commented

2026-04-10 04:53:47 +00:00

Author

Owner

⚠️ Orphan Issue — Parent Epic Required

This issue was created without a parent Epic being specified. Per CONTRIBUTING.md, orphan issues are not permitted — every issue must be linked to a parent Epic via Forgejo's dependency system (child blocks parent).

Action required for maintainers:

Identify the appropriate parent Epic for CI/CD pipeline infrastructure improvements

Create the dependency link so that this issue blocks the parent Epic:

curl -s -X POST "https://git.cleverthis.com/api/v1/repos/cleveragents/cleveragents-core/issues/6889/blocks" \
  -H "Authorization: token <FORGEJO_PAT>" \
  -H "Content-Type: application/json" \
  -d '{"owner": "cleveragents", "repo": "cleveragents-core", "index": <PARENT_EPIC_NUMBER>}'

If no suitable Epic exists, create one before linking this issue.

This issue will remain in State/Unverified until a maintainer triages it and establishes the parent link.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: new-issue-creator

⚠️ **Orphan Issue — Parent Epic Required** This issue was created without a parent Epic being specified. Per `CONTRIBUTING.md`, orphan issues are **not permitted** — every issue must be linked to a parent Epic via Forgejo's dependency system (child **blocks** parent). **Action required for maintainers:** 1. Identify the appropriate parent Epic for CI/CD pipeline infrastructure improvements 2. Create the dependency link so that this issue **blocks** the parent Epic: ```bash curl -s -X POST "https://git.cleverthis.com/api/v1/repos/cleveragents/cleveragents-core/issues/6889/blocks" \ -H "Authorization: token <FORGEJO_PAT>" \ -H "Content-Type: application/json" \ -d '{"owner": "cleveragents", "repo": "cleveragents-core", "index": <PARENT_EPIC_NUMBER>}' ``` 3. If no suitable Epic exists, create one before linking this issue. This issue will remain in `State/Unverified` until a maintainer triages it and establishes the parent link. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: new-issue-creator

HAL9000 referenced this issue

2026-04-10 05:03:18 +00:00

AUTO-PROD-BLDR: Product Builder Session — v3.7.0 Production Ready #6870

HAL9000 self-assigned this

2026-04-10 06:16:35 +00:00

HAL9000 added this to the v3.5.0 milestone

2026-04-10 06:16:35 +00:00

HAL9000 added