TEST-INFRA: [Tooling] Add a tool to retrieve CI execution times #2219

Open
opened 2026-04-03 09:36:56 +00:00 by freemo · 0 comments
Owner

Metadata

  • Branch: feature/test-infra-ci-execution-time-tool
  • Commit Message: feat(ci): add Forgejo API tool to retrieve CI check run durations
  • Milestone: v3.8.0
  • Parent Epic: #1678

Background and Context

Currently, it is not possible to analyze the CI execution time because there is no tool to retrieve the duration of CI checks. This information is essential for identifying performance bottlenecks in the CI/CD pipeline and for validating the improvements introduced by the parent Epic (#1678 — CI Execution Time Optimization).

Without a dedicated tool to query CI check run durations, developers and agents cannot programmatically measure how long individual CI jobs take, making it impossible to detect regressions or confirm that optimization work has had the intended effect.

Expected Behavior

A new Forgejo API tool should be available that allows retrieving the CI check run durations for a given commit or pull request. The tool should return structured data (e.g., job name, start time, end time, duration in seconds) for each check run associated with the queried ref.

Acceptance Criteria

  • A new tool function exists that accepts a commit SHA or pull request number and returns CI check run duration data from the Forgejo API
  • The tool returns at minimum: job name, status, start time, end time, and computed duration (seconds)
  • The tool handles missing or incomplete check run data gracefully (e.g., in-progress or queued runs)
  • The tool is covered by BDD scenarios (Behave/Gherkin) at the unit level
  • The tool is covered by at least one Robot Framework integration smoke test
  • All existing nox stages continue to pass after the addition

Supporting Information

  • Parent Epic: #1678 — CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing
  • Forgejo Check Runs API: GET /api/v1/repos/{owner}/{repo}/commits/{ref}/statuses and GET /api/v1/repos/{owner}/{repo}/check-runs
  • This tool is a prerequisite for any automated CI performance regression detection

Subtasks

  • Research Forgejo API endpoints for retrieving check run / CI status data (statuses vs. check-runs)
  • Implement get_ci_check_run_durations(ref) tool function in the appropriate tooling module
  • Add argument validation (non-empty ref, valid format)
  • Handle edge cases: in-progress runs, queued runs, missing data
  • Tests (Behave): Add BDD scenarios for the new tool (happy path, missing data, invalid ref)
  • Tests (Robot): Add integration smoke test for the new tool against the live Forgejo instance
  • Update relevant documentation (tool reference, README if applicable)
  • Verify coverage >= 97% via nox -s coverage_report
  • Run nox (all default sessions), fix any errors

Definition of Done

This issue is complete when:

  • All subtasks above are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly (feat(ci): add Forgejo API tool to retrieve CI check run durations), followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly (feature/test-infra-ci-execution-time-tool).
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.
  • All nox stages pass.
  • Coverage >= 97%.

Automated by CleverAgents Bot
Supervisor: Unknown | Agent: ca-new-issue-creator

## Metadata - **Branch**: `feature/test-infra-ci-execution-time-tool` - **Commit Message**: `feat(ci): add Forgejo API tool to retrieve CI check run durations` - **Milestone**: v3.8.0 - **Parent Epic**: #1678 ## Background and Context Currently, it is not possible to analyze the CI execution time because there is no tool to retrieve the duration of CI checks. This information is essential for identifying performance bottlenecks in the CI/CD pipeline and for validating the improvements introduced by the parent Epic (#1678 — CI Execution Time Optimization). Without a dedicated tool to query CI check run durations, developers and agents cannot programmatically measure how long individual CI jobs take, making it impossible to detect regressions or confirm that optimization work has had the intended effect. ## Expected Behavior A new Forgejo API tool should be available that allows retrieving the CI check run durations for a given commit or pull request. The tool should return structured data (e.g., job name, start time, end time, duration in seconds) for each check run associated with the queried ref. ## Acceptance Criteria - [ ] A new tool function exists that accepts a commit SHA or pull request number and returns CI check run duration data from the Forgejo API - [ ] The tool returns at minimum: job name, status, start time, end time, and computed duration (seconds) - [ ] The tool handles missing or incomplete check run data gracefully (e.g., in-progress or queued runs) - [ ] The tool is covered by BDD scenarios (Behave/Gherkin) at the unit level - [ ] The tool is covered by at least one Robot Framework integration smoke test - [ ] All existing nox stages continue to pass after the addition ## Supporting Information - Parent Epic: #1678 — CI Execution Time Optimization — Timeouts, Concurrency, and Coverage Artifact Sharing - Forgejo Check Runs API: `GET /api/v1/repos/{owner}/{repo}/commits/{ref}/statuses` and `GET /api/v1/repos/{owner}/{repo}/check-runs` - This tool is a prerequisite for any automated CI performance regression detection ## Subtasks - [ ] Research Forgejo API endpoints for retrieving check run / CI status data (statuses vs. check-runs) - [ ] Implement `get_ci_check_run_durations(ref)` tool function in the appropriate tooling module - [ ] Add argument validation (non-empty ref, valid format) - [ ] Handle edge cases: in-progress runs, queued runs, missing data - [ ] Tests (Behave): Add BDD scenarios for the new tool (happy path, missing data, invalid ref) - [ ] Tests (Robot): Add integration smoke test for the new tool against the live Forgejo instance - [ ] Update relevant documentation (tool reference, README if applicable) - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly (`feat(ci): add Forgejo API tool to retrieve CI check run durations`), followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly (`feature/test-infra-ci-execution-time-tool`). - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. - All nox stages pass. - Coverage >= 97%. --- **Automated by CleverAgents Bot** Supervisor: Unknown | Agent: ca-new-issue-creator
freemo added this to the v3.8.0 milestone 2026-04-03 09:37:31 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Reference
cleveragents/cleveragents-core#2219
No description provided.