Generate JUnit XML reports for test suites to track execution times #1714

Open
opened 2026-04-02 23:33:41 +00:00 by freemo · 1 comment
Owner

Metadata

  • Branch: task/junit-xml-test-reports
  • Commit Message: feat(qa): generate JUnit XML reports for test suites to track execution times
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

Test execution times are currently not captured in a machine-readable format. JUnit XML is the industry-standard report format supported by CI platforms (Forgejo Actions, Jenkins, GitLab CI, etc.) for surfacing per-test timing data, pass/fail status, and suite-level summaries. Without these reports, it is difficult to identify slow tests, track regressions in execution time, or integrate with external test dashboards.

Expected Behavior

All test suites (unit, integration, BDD/Behave, Robot Framework) should emit JUnit XML report files during CI runs. These reports should be stored as CI artifacts and parsed by the CI platform to display per-test timing and results.

Acceptance Criteria

  • All nox test sessions produce JUnit XML output files in a designated reports directory (e.g., reports/junit/)
  • JUnit XML files include per-test execution times, suite names, and pass/fail/error/skip status
  • CI workflow uploads JUnit XML files as artifacts after each test job
  • No existing test behaviour or coverage thresholds are regressed

Supporting Information

  • Parent Epic: #1678 (CI Execution Time Optimization)
  • JUnit XML is natively supported by Forgejo Actions for test result visualization
  • Behave supports --junit flag; pytest supports --junitxml; Robot Framework supports --xunit

Subtasks

  • Configure Behave (BDD) sessions in noxfile.py to emit JUnit XML via --junit --junit-directory reports/junit/
  • Configure pytest sessions in noxfile.py to emit JUnit XML via --junitxml=reports/junit/<suite>.xml
  • Configure Robot Framework sessions in noxfile.py to emit xUnit-compatible XML via --xunit reports/junit/<suite>.xml
  • Create reports/junit/ directory (or ensure it is created at runtime) and add it to .gitignore
  • Update .forgejo/workflows/ci.yml to upload reports/junit/ as a CI artifact after each test job
  • Update features/ci_workflow_validation.feature to assert JUnit artifact upload steps are present
  • Verify all nox stages pass after changes
  • Verify coverage >= 97% via nox -s coverage_report
  • Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (feat(qa): generate JUnit XML reports for test suites to track execution times), followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (task/junit-xml-test-reports).
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

## Metadata - **Branch**: `task/junit-xml-test-reports` - **Commit Message**: `feat(qa): generate JUnit XML reports for test suites to track execution times` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context Test execution times are currently not captured in a machine-readable format. JUnit XML is the industry-standard report format supported by CI platforms (Forgejo Actions, Jenkins, GitLab CI, etc.) for surfacing per-test timing data, pass/fail status, and suite-level summaries. Without these reports, it is difficult to identify slow tests, track regressions in execution time, or integrate with external test dashboards. ## Expected Behavior All test suites (unit, integration, BDD/Behave, Robot Framework) should emit JUnit XML report files during CI runs. These reports should be stored as CI artifacts and parsed by the CI platform to display per-test timing and results. ## Acceptance Criteria - [ ] All nox test sessions produce JUnit XML output files in a designated reports directory (e.g., `reports/junit/`) - [ ] JUnit XML files include per-test execution times, suite names, and pass/fail/error/skip status - [ ] CI workflow uploads JUnit XML files as artifacts after each test job - [ ] No existing test behaviour or coverage thresholds are regressed ## Supporting Information - Parent Epic: #1678 (CI Execution Time Optimization) - JUnit XML is natively supported by Forgejo Actions for test result visualization - Behave supports `--junit` flag; pytest supports `--junitxml`; Robot Framework supports `--xunit` ## Subtasks - [ ] Configure Behave (BDD) sessions in `noxfile.py` to emit JUnit XML via `--junit --junit-directory reports/junit/` - [ ] Configure pytest sessions in `noxfile.py` to emit JUnit XML via `--junitxml=reports/junit/<suite>.xml` - [ ] Configure Robot Framework sessions in `noxfile.py` to emit xUnit-compatible XML via `--xunit reports/junit/<suite>.xml` - [ ] Create `reports/junit/` directory (or ensure it is created at runtime) and add it to `.gitignore` - [ ] Update `.forgejo/workflows/ci.yml` to upload `reports/junit/` as a CI artifact after each test job - [ ] Update `features/ci_workflow_validation.feature` to assert JUnit artifact upload steps are present - [ ] Verify all nox stages pass after changes - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`feat(qa): generate JUnit XML reports for test suites to track execution times`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`task/junit-xml-test-reports`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-02 23:34:02 +00:00
Author
Owner

Issue triaged by project owner:

  • State: Verified
  • MoSCoW: MoSCoW/Could Have — CI/test infrastructure improvement. Could Have.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

Issue triaged by project owner: - **State**: Verified - **MoSCoW**: MoSCoW/Could Have — CI/test infrastructure improvement. Could Have. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#1714
No description provided.