TEST-INFRA: [ci-execution-time] Investigate and optimize the 'test(session): add robot session model smoke tests' workflow #1992

Open
opened 2026-04-03 00:32:19 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: task/ci-execution-time-session-smoke-tests
  • Commit Message: perf(ci): investigate and optimize robot session model smoke tests execution time
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

The test(session): add robot session model smoke tests workflow has been identified as one of the longest-running workflows in the CI pipeline. It has an average execution time of over 1 hour and 13 minutes, which is significantly above acceptable thresholds and contributes to slow feedback loops for developers.

Current Behavior

The test(session): add robot session model smoke tests CI workflow runs for an average of over 1 hour and 13 minutes per execution, making it one of the most time-consuming jobs in the pipeline.

Expected Behavior

The workflow execution time should be reduced by at least 50% (i.e., to under ~37 minutes), achieved through investigation and targeted optimizations such as parallelization, caching, or test splitting.

Acceptance Criteria

  • The execution time of the test(session): add robot session model smoke tests workflow is reduced by at least 50%.
  • All existing test scenarios continue to pass after optimizations are applied.
  • No test coverage is dropped as a result of the changes.

Supporting Information

  • Parent Epic: #1678 (CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing)
  • This issue was surfaced during CI performance analysis as part of the broader CI optimization effort.

Subtasks

  • Profile the test(session): add robot session model smoke tests workflow to identify the root cause of the long execution time
  • Investigate parallelization opportunities (e.g., splitting test suites across parallel jobs or workers)
  • Investigate caching opportunities (e.g., dependency caching, fixture caching, database state reuse)
  • Investigate test splitting strategies (e.g., by tag, by feature file, or by estimated duration)
  • Implement the most impactful optimization(s) identified
  • Validate that all Robot Framework session smoke test scenarios still pass after changes
  • Measure and document the new average execution time to confirm ≥50% reduction
  • Update .forgejo/workflows/ci.yml if workflow-level changes are required
  • Verify coverage ≥ 97% via nox -s coverage_report
  • Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • The test(session): add robot session model smoke tests workflow execution time is reduced by at least 50% (from ~73 minutes to under ~37 minutes), verified by CI run metrics.
  • All Robot Framework session model smoke test scenarios pass without modification to test logic.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/ci-execution-time-session-smoke-tests` - **Commit Message**: `perf(ci): investigate and optimize robot session model smoke tests execution time` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context The `test(session): add robot session model smoke tests` workflow has been identified as one of the longest-running workflows in the CI pipeline. It has an average execution time of over **1 hour and 13 minutes**, which is significantly above acceptable thresholds and contributes to slow feedback loops for developers. ## Current Behavior The `test(session): add robot session model smoke tests` CI workflow runs for an average of over 1 hour and 13 minutes per execution, making it one of the most time-consuming jobs in the pipeline. ## Expected Behavior The workflow execution time should be reduced by at least **50%** (i.e., to under ~37 minutes), achieved through investigation and targeted optimizations such as parallelization, caching, or test splitting. ## Acceptance Criteria - The execution time of the `test(session): add robot session model smoke tests` workflow is reduced by at least 50%. - All existing test scenarios continue to pass after optimizations are applied. - No test coverage is dropped as a result of the changes. ## Supporting Information - Parent Epic: #1678 (CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing) - This issue was surfaced during CI performance analysis as part of the broader CI optimization effort. ## Subtasks - [ ] Profile the `test(session): add robot session model smoke tests` workflow to identify the root cause of the long execution time - [ ] Investigate parallelization opportunities (e.g., splitting test suites across parallel jobs or workers) - [ ] Investigate caching opportunities (e.g., dependency caching, fixture caching, database state reuse) - [ ] Investigate test splitting strategies (e.g., by tag, by feature file, or by estimated duration) - [ ] Implement the most impactful optimization(s) identified - [ ] Validate that all Robot Framework session smoke test scenarios still pass after changes - [ ] Measure and document the new average execution time to confirm ≥50% reduction - [ ] Update `.forgejo/workflows/ci.yml` if workflow-level changes are required - [ ] Verify coverage ≥ 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - The `test(session): add robot session model smoke tests` workflow execution time is reduced by at least 50% (from ~73 minutes to under ~37 minutes), verified by CI run metrics. - All Robot Framework session model smoke test scenarios pass without modification to test logic. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-03 00:32:36 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • Priority: High — A 73-minute CI job is a severe developer productivity bottleneck. Reducing this by 50% would significantly improve feedback loops.
  • Milestone: v3.8.0 (Server Implementation — already assigned. While this is CI infrastructure, the parent epic #1678 is correctly scoped to v3.8.0.)
  • MoSCoW: Could Have — CI execution time optimization is important for developer productivity but does not block any milestone deliverables. The project should focus on M1-M6 feature work first. This should be picked up when there is bandwidth.
  • Parent Epic: #1678 (CI Execution Time Optimization — already linked, confirmed correct)

Well-documented with clear acceptance criteria (≥50% reduction target) and investigation approach.


Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **Priority**: High — A 73-minute CI job is a severe developer productivity bottleneck. Reducing this by 50% would significantly improve feedback loops. - **Milestone**: v3.8.0 (Server Implementation — already assigned. While this is CI infrastructure, the parent epic #1678 is correctly scoped to v3.8.0.) - **MoSCoW**: Could Have — CI execution time optimization is important for developer productivity but does not block any milestone deliverables. The project should focus on M1-M6 feature work first. This should be picked up when there is bandwidth. - **Parent Epic**: #1678 (CI Execution Time Optimization — already linked, confirmed correct) Well-documented with clear acceptance criteria (≥50% reduction target) and investigation approach. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cleveragents/cleveragents-core#1992
No description provided.