cleveragents/cleveragents-core

Fork 3

TEST-INFRA: [ci-execution-time] Increase default parallelization for unit and integration tests #2285

New issue

Closed

opened 2026-04-03 12:50:46 +00:00 by freemo · 3 comments

freemo commented

2026-04-03 12:50:46 +00:00

Owner

Metadata

Branch: task/ci-execution-time-increase-default-parallelization
Commit Message: perf(nox): increase default parallelization for unit and integration tests
Milestone: v3.8.0
Parent Epic: #1678

Summary

The noxfile.py currently defaults to a very conservative number of parallel processes (min(os.cpu_count(), 2)) for both Behave unit tests and Robot integration tests. While this is a safe default, it likely underutilizes the resources of modern CI runners, which often have 4, 8, or more CPU cores available. This conservative setting can lead to unnecessarily long test execution times.

Proposal

To improve CI performance, we should increase the default level of test parallelization.

Subtasks

Analyze the typical CPU and memory resources of the Forgejo CI runners used by this project.
Determine an optimal, higher default number of processes for the _default_processes function in noxfile.py. A value of 4 or 8 might be a good starting point.
Alternatively, keep the default low but explicitly set the TEST_PROCESSES environment variable to a higher value within the CI workflow configuration.
Monitor the stability and execution time of the test suites after the change to ensure there are no resource contention issues.
Run nox (all default sessions), fix any errors.
Verify coverage >=97% via nox -s coverage_report.

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
The default parallelization for tests is increased, resulting in a measurable decrease in the execution time of the unit_tests and integration_tests nox sessions.
The CI pipeline remains stable and reliable.
A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
All nox stages pass.
Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-execution-time-increase-default-parallelization` - **Commit Message**: `perf(nox): increase default parallelization for unit and integration tests` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Summary The `noxfile.py` currently defaults to a very conservative number of parallel processes (`min(os.cpu_count(), 2)`) for both Behave unit tests and Robot integration tests. While this is a safe default, it likely underutilizes the resources of modern CI runners, which often have 4, 8, or more CPU cores available. This conservative setting can lead to unnecessarily long test execution times. ## Proposal To improve CI performance, we should increase the default level of test parallelization. ## Subtasks - [ ] Analyze the typical CPU and memory resources of the Forgejo CI runners used by this project. - [ ] Determine an optimal, higher default number of processes for the `_default_processes` function in `noxfile.py`. A value of 4 or 8 might be a good starting point. - [ ] Alternatively, keep the default low but explicitly set the `TEST_PROCESSES` environment variable to a higher value within the CI workflow configuration. - [ ] Monitor the stability and execution time of the test suites after the change to ensure there are no resource contention issues. - [ ] Run `nox` (all default sessions), fix any errors. - [ ] Verify coverage >=97% via `nox -s coverage_report`. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - The default parallelization for tests is increased, resulting in a measurable decrease in the execution time of the `unit_tests` and `integration_tests` nox sessions. - The CI pipeline remains stable and reliable. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

freemo added this to the v3.8.0 milestone

2026-04-03 12:51:45 +00:00

freemo added a new dependency

2026-04-03 12:53:29 +00:00

#1678 Epic: CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing

freemo commented

2026-04-03 12:58:13 +00:00

Author

Owner

Issue triaged by project owner:

State: Verified
Priority: Medium (confirmed)
Milestone: v3.8.0 (confirmed — CI infrastructure)
MoSCoW: Could Have — Increasing default parallelization is a CI performance tuning task. Not blocking any deliverables.
Parent Epic: #1678 (confirmed correct)

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: Medium (confirmed) - **Milestone**: v3.8.0 (confirmed — CI infrastructure) - **MoSCoW**: Could Have — Increasing default parallelization is a CI performance tuning task. Not blocking any deliverables. - **Parent Epic**: #1678 (confirmed correct) --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo added the

MoSCoW

Could have

label

2026-04-03 12:58:15 +00:00

freemo commented

2026-04-03 12:58:16 +00:00

Author

Owner

⚠️ Potential duplicate: Issue #2277 ("Increase default parallelism for unit and integration tests") and #2283 ("Increase default parallelism for unit and integration tests") describe the same work. Please check if this issue is a duplicate before proceeding.

Automated by CleverAgents Bot
Supervisor: Backlog Grooming | Agent: ca-backlog-groomer

⚠️ **Potential duplicate**: Issue #2277 ("Increase default parallelism for unit and integration tests") and #2283 ("Increase default parallelism for unit and integration tests") describe the same work. Please check if this issue is a duplicate before proceeding. --- **Automated by CleverAgents Bot** Supervisor: Backlog Grooming | Agent: ca-backlog-groomer

freemo referenced this issue

2026-04-03 12:58:39 +00:00

[Automated] Product Build Session State — v3.7.0 Production Sprint #2006

freemo referenced this issue

2026-04-03 14:27:28 +00:00

TEST-INFRA: [ci-execution-time] Make test parallelism level more configurable #2326

freemo referenced this issue

2026-04-03 15:02:56 +00:00

TEST-INFRA: [ci-execution-time] Increase default parallelism for unit_tests and integration_tests #2342

freemo commented

2026-04-03 17:22:55 +00:00

Author

Owner

Closing as duplicate of #2333.

This issue ("Increase default parallelization for unit and integration tests") is nearly identical to #2283 and #2333. Consolidating under #2333 to avoid duplicate work.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Closing as duplicate of #2333. This issue ("Increase default parallelization for unit and integration tests") is nearly identical to #2283 and #2333. Consolidating under #2333 to avoid duplicate work. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner

freemo closed this issue

2026-04-03 17:23:09 +00:00

freemo referenced this issue

2026-04-03 17:23:59 +00:00

Proposal: improve ca-test-infra-improver — strengthen duplicate avoidance to prevent near-identical issues #2370

freemo referenced this issue

2026-04-03 17:24:35 +00:00

TEST-INFRA: [ci-execution-time] Investigate and increase test parallelism #2333