TEST-INFRA: [flaky-tests] Investigate consistently failing tests #5796

New issue

Open

opened 2026-04-09 09:45:34 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-09 09:45:34 +00:00

Owner

Metadata

Branch: test/investigate-consistently-failing-tests
Commit Message: test(ci): investigate and fix consistently failing Robot Framework tests
Milestone: (backlog — see note below)
Parent Epic: #5407

Background

During CI log analysis for flaky tests, no flaky tests were found. However, the following tests are consistently failing and require investigation and remediation:

Test Suite	Test Case	Log File
`Robot.Coverage Threshold`	`Noxfile Contains Coverage Threshold Constant`	`pr4583_integration_tests_run12211_job5.log`
`Robot.Actor Registry Persistence`	`Actor Namespaced Name Validation`	`pr_5271_integration_tests.log`
`Robot.Actor Add Yaml First Path`	`Actor Add Preserves yaml_text Via Registry upsert_actor`	`pr_5271_integration_tests.log`

These tests are not flaky (intermittent) — they are consistently failing, which indicates a real defect in either the test or the production code.

Subtasks

Reproduce each failing test locally using nox -e integration_tests
Investigate Robot.Coverage Threshold → Noxfile Contains Coverage Threshold Constant failure root cause
Investigate Robot.Actor Registry Persistence → Actor Namespaced Name Validation failure root cause
Investigate Robot.Actor Add Yaml First Path → Actor Add Preserves yaml_text Via Registry upsert_actor failure root cause
Determine if failures are due to test defects or production code defects
Fix each failing test (update test or fix production code as appropriate)
Verify all three tests pass consistently in CI
Ensure no regressions introduced by the fixes

Definition of Done

All three identified consistently failing tests pass in CI
Root cause of each failure is documented in a PR comment
No new test failures introduced
All nox stages pass
Coverage >= 97%

Backlog note: This issue was discovered during autonomous operation
on milestone v3.8.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure | Agent: new-issue-creator

## Metadata - **Branch**: `test/investigate-consistently-failing-tests` - **Commit Message**: `test(ci): investigate and fix consistently failing Robot Framework tests` - **Milestone**: *(backlog — see note below)* - **Parent Epic**: #5407 ## Background During CI log analysis for flaky tests, no flaky tests were found. However, the following tests are **consistently failing** and require investigation and remediation: | Test Suite | Test Case | Log File | |---|---|---| | `Robot.Coverage Threshold` | `Noxfile Contains Coverage Threshold Constant` | `pr4583_integration_tests_run12211_job5.log` | | `Robot.Actor Registry Persistence` | `Actor Namespaced Name Validation` | `pr_5271_integration_tests.log` | | `Robot.Actor Add Yaml First Path` | `Actor Add Preserves yaml_text Via Registry upsert_actor` | `pr_5271_integration_tests.log` | These tests are not flaky (intermittent) — they are **consistently failing**, which indicates a real defect in either the test or the production code. ## Subtasks - [ ] Reproduce each failing test locally using `nox -e integration_tests` - [ ] Investigate `Robot.Coverage Threshold` → `Noxfile Contains Coverage Threshold Constant` failure root cause - [ ] Investigate `Robot.Actor Registry Persistence` → `Actor Namespaced Name Validation` failure root cause - [ ] Investigate `Robot.Actor Add Yaml First Path` → `Actor Add Preserves yaml_text Via Registry upsert_actor` failure root cause - [ ] Determine if failures are due to test defects or production code defects - [ ] Fix each failing test (update test or fix production code as appropriate) - [ ] Verify all three tests pass consistently in CI - [ ] Ensure no regressions introduced by the fixes ## Definition of Done - [ ] All three identified consistently failing tests pass in CI - [ ] Root cause of each failure is documented in a PR comment - [ ] No new test failures introduced - [ ] All nox stages pass - [ ] Coverage >= 97% > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.8.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure | Agent: new-issue-creator

Rows
Columns