Implement @tdd_expected_fail tag handling in Robot Framework #628

Closed
opened 2026-03-07 21:32:10 +00:00 by freemo · 9 comments
Owner

Metadata

  • Commit Message: feat(testing): implement @tdd_expected_fail tag handling in Robot Framework
  • Branch: feature/m5-robot-tdd-tags

Background and Context

The project's Bug Fix Workflow (see CONTRIBUTING.md) requires a three-tag system for TDD bug-capture tests. This tag system must work in both Behave (unit tests) and Robot Framework (integration tests). Issue #627 implements the Behave side; this issue implements the Robot Framework side.

The three tags are:

  • tdd_bug — generic filter tag, always present on TDD bug tests
  • tdd_bug_<N> — links to specific bug issue number, always present
  • tdd_expected_fail — behavioral switch that inverts test pass/fail

Robot Framework must handle these tags so that:

  • When tdd_expected_fail is present, a test that fails (bug still exists) is reported as passed
  • When tdd_expected_fail is present, a test that passes (bug was fixed without removing the tag) is reported as failed with a clear error message
  • Tag validation rules are enforced: tdd_bug_<N> requires tdd_bug, and tdd_expected_fail requires both

Note: Robot Framework uses [Tags] syntax without @ prefix, but the tag names are the same.

Expected Behavior

Robot Framework listener or pre-run modifier in robot/ correctly handles the tdd_expected_fail tag by inverting test results for tagged tests, and enforces tag consistency rules.

Acceptance Criteria

  • Tests tagged tdd_expected_fail that fail have their result inverted to pass (expected failure).
  • Tests tagged tdd_expected_fail that pass have their result inverted to fail with message: "Bug appears to be fixed. Remove the tdd_expected_fail tag and verify the fix through the bug fix workflow."
  • A test with tdd_bug_<N> but missing tdd_bug causes a clear validation error.
  • A test with tdd_expected_fail but missing tdd_bug or tdd_bug_<N> causes a clear validation error.
  • The tdd_bug tag can be used to filter/list all TDD bug tests (e.g., --include tdd_bug).
  • Normal (non-tagged) test execution is completely unaffected.

Definition of Done

This issue is complete when:

  • All subtasks below are completed and checked off.
  • A Git commit is created where the first line of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation.
  • The commit is pushed to the remote on the branch matching the Branch in Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, and merged before this issue is marked done.

Subtasks

  • Code: Implement a Robot Framework listener or pre-run modifier that detects the tdd_expected_fail tag and inverts the test's pass/fail result. Place it in the appropriate location under robot/ or config/.
  • Code: Implement tag validation: if tdd_bug_<N> is present, verify tdd_bug is also present. If tdd_expected_fail is present, verify both tdd_bug and at least one tdd_bug_<N> are present. Raise a clear error if validation fails.
  • Code: When a tdd_expected_fail test unexpectedly passes, set the failure message to: "Bug appears to be fixed. Remove the tdd_expected_fail tag from this test and verify the fix through the bug fix workflow. See CONTRIBUTING.md > Bug Fix Workflow."
  • Code: Register the listener/modifier in the nox integration_tests session configuration so it is always active during Robot test runs.
  • Code: Ensure the tag handling does not interfere with Robot Framework's normal reporting, log generation, or other listeners.
  • Docs: Add inline documentation explaining the three-tag system and referencing CONTRIBUTING.md > TDD Bug Test Tags.
  • Tests (Behave): N/A — this is Robot Framework-specific infrastructure.
  • Tests (Robot): Add Robot test cases that verify: (a) a tdd_expected_fail test that fails is reported as passed, (b) a tdd_expected_fail test that passes is reported as failed, (c) tag validation catches missing tdd_bug, (d) non-tagged tests are unaffected.
  • Tests (ASV): N/A — no performance-sensitive code.
  • Quality: Verify coverage >=97% via nox -s coverage_report. If coverage is <97% then review the current unit test coverage report at build/coverage.xml and use it to write new Behave based unit tests to improve code coverage. Specifically, write Behave style unit tests that are descriptively named and specifically improve coverage on whichever file has the most uncovered lines by writing tests that will target the uncovered lines in the report. Once that is done rerun nox -s coverage_report to verify all tests pass and coverage is above >=97%. Only mark this as complete once coverage is >=97%, if not repeat this task as many times as is needed until coverage reaches >=97%.
  • Quality: Run nox (all default sessions, including benchmark), fix any errors if needed ensuring nox passes across entire code base, do not ignore any failure even if it seems unrelated to this commit, fix it.
## Metadata - **Commit Message**: `feat(testing): implement @tdd_expected_fail tag handling in Robot Framework` - **Branch**: `feature/m5-robot-tdd-tags` ## Background and Context The project's Bug Fix Workflow (see `CONTRIBUTING.md`) requires a three-tag system for TDD bug-capture tests. This tag system must work in both Behave (unit tests) and Robot Framework (integration tests). Issue #627 implements the Behave side; this issue implements the Robot Framework side. The three tags are: - `tdd_bug` — generic filter tag, always present on TDD bug tests - `tdd_bug_<N>` — links to specific bug issue number, always present - `tdd_expected_fail` — behavioral switch that inverts test pass/fail Robot Framework must handle these tags so that: - When `tdd_expected_fail` is present, a test that **fails** (bug still exists) is reported as **passed** - When `tdd_expected_fail` is present, a test that **passes** (bug was fixed without removing the tag) is reported as **failed** with a clear error message - Tag validation rules are enforced: `tdd_bug_<N>` requires `tdd_bug`, and `tdd_expected_fail` requires both Note: Robot Framework uses `[Tags]` syntax without `@` prefix, but the tag names are the same. ## Expected Behavior Robot Framework listener or pre-run modifier in `robot/` correctly handles the `tdd_expected_fail` tag by inverting test results for tagged tests, and enforces tag consistency rules. ## Acceptance Criteria - [x] Tests tagged `tdd_expected_fail` that fail have their result inverted to pass (expected failure). - [x] Tests tagged `tdd_expected_fail` that pass have their result inverted to fail with message: "Bug appears to be fixed. Remove the tdd_expected_fail tag and verify the fix through the bug fix workflow." - [x] A test with `tdd_bug_<N>` but missing `tdd_bug` causes a clear validation error. - [x] A test with `tdd_expected_fail` but missing `tdd_bug` or `tdd_bug_<N>` causes a clear validation error. - [x] The `tdd_bug` tag can be used to filter/list all TDD bug tests (e.g., `--include tdd_bug`). - [x] Normal (non-tagged) test execution is completely unaffected. ## Definition of Done This issue is complete when: - All subtasks below are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the implementation. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, and **merged** before this issue is marked done. ## Subtasks - [x] Code: Implement a Robot Framework listener or pre-run modifier that detects the `tdd_expected_fail` tag and inverts the test's pass/fail result. Place it in the appropriate location under `robot/` or `config/`. - [x] Code: Implement tag validation: if `tdd_bug_<N>` is present, verify `tdd_bug` is also present. If `tdd_expected_fail` is present, verify both `tdd_bug` and at least one `tdd_bug_<N>` are present. Raise a clear error if validation fails. - [x] Code: When a `tdd_expected_fail` test unexpectedly passes, set the failure message to: "Bug appears to be fixed. Remove the tdd_expected_fail tag from this test and verify the fix through the bug fix workflow. See CONTRIBUTING.md > Bug Fix Workflow." - [x] Code: Register the listener/modifier in the nox `integration_tests` session configuration so it is always active during Robot test runs. - [x] Code: Ensure the tag handling does not interfere with Robot Framework's normal reporting, log generation, or other listeners. - [x] Docs: Add inline documentation explaining the three-tag system and referencing `CONTRIBUTING.md > TDD Bug Test Tags`. - [x] Tests (Behave): N/A — this is Robot Framework-specific infrastructure. - [x] Tests (Robot): Add Robot test cases that verify: (a) a `tdd_expected_fail` test that fails is reported as passed, (b) a `tdd_expected_fail` test that passes is reported as failed, (c) tag validation catches missing `tdd_bug`, (d) non-tagged tests are unaffected. - [x] Tests (ASV): N/A — no performance-sensitive code. - [x] Quality: Verify coverage >=97% via `nox -s coverage_report`. If coverage is <97% then review the current unit test coverage report at `build/coverage.xml` and use it to write new Behave based unit tests to improve code coverage. Specifically, write Behave style unit tests that are descriptively named and specifically improve coverage on whichever file has the most uncovered lines by writing tests that will target the uncovered lines in the report. Once that is done rerun `nox -s coverage_report` to verify all tests pass and coverage is above >=97%. Only mark this as complete once coverage is >=97%, if not repeat this task as many times as is needed until coverage reaches >=97%. - [x] Quality: Run `nox` (all default sessions, including benchmark), fix any errors if needed ensuring nox passes across **entire** code base, do not ignore any failure even if it seems unrelated to this commit, fix it.
freemo added this to the v3.4.0 milestone 2026-03-07 23:06:37 +00:00
Author
Owner

Triage — Day 27

Status: Triaged and labeled.

Actions Taken

  • Added labels: Type/Feature, Priority/High, MoSCoW/Must have, State/Unverified, Points/5
  • Assigned to: @brent.edwards (QA lead, most familiar with test infrastructure)
  • Milestone: v3.4.0 (M5 — TDD infrastructure)

Context

Robot Framework counterpart to #627 (Behave). Same blocking relationship — all TDD counterpart issues (#630-#638) depend on this for CI to handle tdd_expected_fail tags correctly.

Can be developed in parallel with #627 since they are independent (Behave vs Robot Framework).

## Triage — Day 27 **Status**: Triaged and labeled. ### Actions Taken - Added labels: `Type/Feature`, `Priority/High`, `MoSCoW/Must have`, `State/Unverified`, `Points/5` - Assigned to: @brent.edwards (QA lead, most familiar with test infrastructure) - Milestone: v3.4.0 (M5 — TDD infrastructure) ### Context Robot Framework counterpart to #627 (Behave). Same blocking relationship — all TDD counterpart issues (#630-#638) depend on this for CI to handle `tdd_expected_fail` tags correctly. Can be developed in parallel with #627 since they are independent (Behave vs Robot Framework).
freemo modified the milestone from v3.4.0 to v3.2.0 2026-03-09 20:09:25 +00:00
Author
Owner

PM Note (Day 29) — Reassignment and Milestone Change

Changes:

  • Assignee: @brent.edwards → @hurui200320
  • Milestone: v3.4.0 → v3.2.0
  • State: Unverified → Verified

Rationale: Robot Framework @tdd_expected_fail tag support is the counterpart to #627 (Behave). Rui has capacity (was at 3 SP) and this provides meaningful, focused work on the test infrastructure side.

@hurui200320 — This involves adding @tdd_expected_fail tag support to the Robot Framework test configuration. Tests tagged with this marker should be expected to fail (proving a bug exists) and should pass the test suite without being treated as regressions. See CONTRIBUTING.md's Bug Fix Workflow section and coordinate with @CoreRasurae who is doing the Behave equivalent (#627).

**PM Note (Day 29) — Reassignment and Milestone Change** **Changes:** - **Assignee**: @brent.edwards → @hurui200320 - **Milestone**: v3.4.0 → **v3.2.0** - **State**: Unverified → **Verified** **Rationale:** Robot Framework `@tdd_expected_fail` tag support is the counterpart to #627 (Behave). Rui has capacity (was at 3 SP) and this provides meaningful, focused work on the test infrastructure side. @hurui200320 — This involves adding `@tdd_expected_fail` tag support to the Robot Framework test configuration. Tests tagged with this marker should be expected to fail (proving a bug exists) and should pass the test suite without being treated as regressions. See CONTRIBUTING.md's Bug Fix Workflow section and coordinate with @CoreRasurae who is doing the Behave equivalent (#627).
Member

Implementation Notes — Robot Framework TDD Tag Handling

Design Decisions

Listener v3 API chosen over pre-run modifier.
The Robot Framework Listener v3 API was selected because it provides clean hooks for both tag validation (start_test) and result inversion (end_test). A pre-run modifier would only allow tag validation but not result inversion, which is the core requirement. The Listener v3 API allows direct modification of the result object in end_test, making the inversion straightforward.

Module-level functions instead of a class.
Following the Robot Framework convention for Listener v3, the listener is implemented as module-level start_test and end_test functions with ROBOT_LISTENER_API_VERSION = 3 at module level. This is simpler and sufficient for our needs.

Validation errors stored in _validation_errors dict.
Tag validation happens in start_test but cannot directly fail the test at that point. Instead, validation errors are stored keyed by test longname and applied in end_test to override whatever result the test produced. This ensures the validation error message is the one visible in reports.

Fixture files tagged with tdd_fixture.
The test fixture .robot files under robot/fixtures/ are designed to be run by the helper as sub-processes. They would cause false failures if picked up by the main pabot runner. Adding tdd_fixture tag and --exclude tdd_fixture in nox prevents this while keeping fixtures co-located with the listener.

Key Code Locations

  • Listener: robot.tdd_expected_fail_listener_validate_tdd_tags(), _should_invert_result(), start_test(), end_test()
  • Robot tests: robot.tdd_tag_validation — 5 test cases verifying all acceptance criteria
  • Test helper: robot.helper_tdd_tag_validation — runs fixture .robot files as sub-processes with the listener
  • Fixtures: robot.fixtures.tdd_expected_fail_fails, tdd_expected_fail_passes, tdd_missing_tdd_bug, tdd_expected_fail_missing_bug_n, tdd_normal_test
  • Nox registration: noxfile.integration_tests and noxfile.slow_integration_tests--listener and --exclude tdd_fixture flags added

Alignment with Behave Implementation (PR !665, Issue #627)

The Robot listener mirrors the Behave implementation in features/environment.py:

  • Same tag names (tdd_bug, tdd_bug_<N>, tdd_expected_fail)
  • Same validation rules (identical regex tdd_bug_\d+)
  • Same failure message for unexpected passes
  • Same logging pattern via cleveragents.testing.robot_tdd_tags logger

Test Results (Pre-commit)

  • robot/tdd_tag_validation.robot: 5 tests, 5 passed, 0 failed
  • All 1,361 existing integration tests pass with the listener active
  • Fixture files correctly excluded from main pabot run via tdd_fixture tag
## Implementation Notes — Robot Framework TDD Tag Handling ### Design Decisions **Listener v3 API chosen over pre-run modifier.** The Robot Framework Listener v3 API was selected because it provides clean hooks for both tag validation (`start_test`) and result inversion (`end_test`). A pre-run modifier would only allow tag validation but not result inversion, which is the core requirement. The Listener v3 API allows direct modification of the `result` object in `end_test`, making the inversion straightforward. **Module-level functions instead of a class.** Following the Robot Framework convention for Listener v3, the listener is implemented as module-level `start_test` and `end_test` functions with `ROBOT_LISTENER_API_VERSION = 3` at module level. This is simpler and sufficient for our needs. **Validation errors stored in `_validation_errors` dict.** Tag validation happens in `start_test` but cannot directly fail the test at that point. Instead, validation errors are stored keyed by test longname and applied in `end_test` to override whatever result the test produced. This ensures the validation error message is the one visible in reports. **Fixture files tagged with `tdd_fixture`.** The test fixture `.robot` files under `robot/fixtures/` are designed to be run by the helper as sub-processes. They would cause false failures if picked up by the main pabot runner. Adding `tdd_fixture` tag and `--exclude tdd_fixture` in nox prevents this while keeping fixtures co-located with the listener. ### Key Code Locations - **Listener**: `robot.tdd_expected_fail_listener` — `_validate_tdd_tags()`, `_should_invert_result()`, `start_test()`, `end_test()` - **Robot tests**: `robot.tdd_tag_validation` — 5 test cases verifying all acceptance criteria - **Test helper**: `robot.helper_tdd_tag_validation` — runs fixture .robot files as sub-processes with the listener - **Fixtures**: `robot.fixtures.tdd_expected_fail_fails`, `tdd_expected_fail_passes`, `tdd_missing_tdd_bug`, `tdd_expected_fail_missing_bug_n`, `tdd_normal_test` - **Nox registration**: `noxfile.integration_tests` and `noxfile.slow_integration_tests` — `--listener` and `--exclude tdd_fixture` flags added ### Alignment with Behave Implementation (PR !665, Issue #627) The Robot listener mirrors the Behave implementation in `features/environment.py`: - Same tag names (`tdd_bug`, `tdd_bug_<N>`, `tdd_expected_fail`) - Same validation rules (identical regex `tdd_bug_\d+`) - Same failure message for unexpected passes - Same logging pattern via `cleveragents.testing.robot_tdd_tags` logger ### Test Results (Pre-commit) - `robot/tdd_tag_validation.robot`: 5 tests, 5 passed, 0 failed - All 1,361 existing integration tests pass with the listener active - Fixture files correctly excluded from main pabot run via `tdd_fixture` tag
Member

Implementation Complete — PR #673 Submitted

Summary

All subtasks are complete and checked off. PR #673 (feature/m5-robot-tdd-tagsmaster) implements the full three-tag TDD bug-capture system for Robot Framework integration tests.

Files Changed (10 files, +558 lines)

File Purpose
robot/tdd_expected_fail_listener.py Listener v3 module with start_test (tag validation) and end_test (result inversion)
robot/tdd_tag_validation.robot 5 Robot test cases covering all acceptance criteria
robot/helper_tdd_tag_validation.py Helper script that runs fixtures as sub-processes and inspects output XML
robot/fixtures/tdd_expected_fail_fails.robot Fixture: deliberately failing tdd_expected_fail test
robot/fixtures/tdd_expected_fail_passes.robot Fixture: unexpectedly passing tdd_expected_fail test
robot/fixtures/tdd_missing_tdd_bug.robot Fixture: tdd_bug_N without tdd_bug
robot/fixtures/tdd_expected_fail_missing_bug_n.robot Fixture: tdd_expected_fail + tdd_bug without tdd_bug_N
robot/fixtures/tdd_normal_test.robot Fixture: normal test with no TDD tags
noxfile.py Registered listener via --listener and excluded fixtures via --exclude tdd_fixture
CHANGELOG.md Added changelog entry

Quality Gate Results

Gate Result
nox -s lint PASS
nox -s typecheck PASS (0 errors)
nox -s unit_tests PASS (9813 scenarios, 352 features)
nox -s integration_tests PASS (1367 tests, 0 failures)
nox -s coverage_report PASS (98% >= 97% threshold)

Design Decisions (continued from previous comment)

  • Fixture isolation via tdd_fixture tag: Test fixture .robot files are tagged tdd_fixture and excluded from the main pabot runner via --exclude tdd_fixture in nox. This keeps fixtures co-located with the listener while preventing them from running as part of the normal integration test suite.

  • Sub-process testing approach: The Robot test cases (tdd_tag_validation.robot) run fixture files as sub-processes via a helper Python script. This is necessary because the listener modifies test results within the running Robot process — to verify the listener's behavior, we need to run Robot with the listener active and then inspect the output XML externally. This mirrors how other integration tests in the project work (e.g., cli_formats.robot uses a helper script to run CLI commands and check output).

  • Commit hash: b882b5e208ed0a2e4efc2b7d63a85e4ef5d741a8

Awaiting

  • CI pipeline checks on PR #673
  • Peer review
## Implementation Complete — PR #673 Submitted ### Summary All subtasks are complete and checked off. PR #673 (`feature/m5-robot-tdd-tags` → `master`) implements the full three-tag TDD bug-capture system for Robot Framework integration tests. ### Files Changed (10 files, +558 lines) | File | Purpose | |------|---------| | `robot/tdd_expected_fail_listener.py` | Listener v3 module with `start_test` (tag validation) and `end_test` (result inversion) | | `robot/tdd_tag_validation.robot` | 5 Robot test cases covering all acceptance criteria | | `robot/helper_tdd_tag_validation.py` | Helper script that runs fixtures as sub-processes and inspects output XML | | `robot/fixtures/tdd_expected_fail_fails.robot` | Fixture: deliberately failing tdd_expected_fail test | | `robot/fixtures/tdd_expected_fail_passes.robot` | Fixture: unexpectedly passing tdd_expected_fail test | | `robot/fixtures/tdd_missing_tdd_bug.robot` | Fixture: tdd_bug_N without tdd_bug | | `robot/fixtures/tdd_expected_fail_missing_bug_n.robot` | Fixture: tdd_expected_fail + tdd_bug without tdd_bug_N | | `robot/fixtures/tdd_normal_test.robot` | Fixture: normal test with no TDD tags | | `noxfile.py` | Registered listener via `--listener` and excluded fixtures via `--exclude tdd_fixture` | | `CHANGELOG.md` | Added changelog entry | ### Quality Gate Results | Gate | Result | |------|--------| | `nox -s lint` | PASS | | `nox -s typecheck` | PASS (0 errors) | | `nox -s unit_tests` | PASS (9813 scenarios, 352 features) | | `nox -s integration_tests` | PASS (1367 tests, 0 failures) | | `nox -s coverage_report` | PASS (98% >= 97% threshold) | ### Design Decisions (continued from previous comment) - **Fixture isolation via `tdd_fixture` tag**: Test fixture `.robot` files are tagged `tdd_fixture` and excluded from the main pabot runner via `--exclude tdd_fixture` in nox. This keeps fixtures co-located with the listener while preventing them from running as part of the normal integration test suite. - **Sub-process testing approach**: The Robot test cases (`tdd_tag_validation.robot`) run fixture files as sub-processes via a helper Python script. This is necessary because the listener modifies test results *within* the running Robot process — to verify the listener's behavior, we need to run Robot with the listener active and then inspect the output XML externally. This mirrors how other integration tests in the project work (e.g., `cli_formats.robot` uses a helper script to run CLI commands and check output). - **Commit hash**: b882b5e208ed0a2e4efc2b7d63a85e4ef5d741a8 ### Awaiting - CI pipeline checks on PR #673 - Peer review
Author
Owner

PM Acknowledgment (Day 31):

Thank you @hurui200320. PR #673 submitted with self-review fixes — good practice.

Status: PR #673 has 4 comments (self-review) and a merge conflict. This is on the critical path — it feeds into the CI quality gate (#629) alongside #627.

Action needed:

  1. Rebase PR #673 against current develop to resolve merge conflict
  2. The 17 PR merges from Days 30-31 significantly changed the develop branch — expect non-trivial rebase

Priority: HIGH — this blocks the TDD CI quality gate.

**PM Acknowledgment (Day 31)**: Thank you @hurui200320. PR #673 submitted with self-review fixes — good practice. **Status**: PR #673 has 4 comments (self-review) and a **merge conflict**. This is on the critical path — it feeds into the CI quality gate (#629) alongside #627. **Action needed**: 1. Rebase PR #673 against current `develop` to resolve merge conflict 2. The 17 PR merges from Days 30-31 significantly changed the develop branch — expect non-trivial rebase **Priority**: HIGH — this blocks the TDD CI quality gate.
Member

Rebase onto latest master (2026-03-11)

Rebased feature/m5-robot-tdd-tags onto current master (82def111) to resolve the merge conflict that made PR !673 unmergeable.

Conflicts resolved

Three files had conflicts due to parallel work on master:

  1. CHANGELOG.md — Master added entries for issues #631 and #630 (TDD failing tests for session list/create DI bugs). Our #628 entry was placed ahead of those entries in the Unreleased section.

  2. noxfile.py — Master independently added --listener robot/tdd_expected_fail_listener.py (as a string literal) to both integration_tests and slow_integration_tests. Our branch already had this via the tdd_listener variable (resolved absolute path). Resolved by keeping our variable-based approach and removing the duplicate from master. Also preserved our --exclude tdd_fixture flag that master didn't have.

  3. robot/tdd_expected_fail_listener.py — Master had a simpler version (only end_test, no tag validation, no start_test, no logging, Any types). Our comprehensive version (full tag validation, proper Robot Framework types, logging, docstrings) replaces it entirely, as this is the issue that was meant to implement the full listener per the acceptance criteria.

Quality gates (all pass on rebased branch)

  • nox -s lint: PASS
  • nox -s typecheck: PASS (0 errors, 1 warning)
  • nox -s unit_tests: PASS (10,431 scenarios, 370 features)
  • nox -s integration_tests: PASS (1,457 tests)
  • nox -s coverage_report: PASS (98% >= 97% threshold)

PR !673 is now mergeable (merge_base matches current master HEAD).

## Rebase onto latest master (2026-03-11) Rebased `feature/m5-robot-tdd-tags` onto current master (`82def111`) to resolve the merge conflict that made PR !673 unmergeable. ### Conflicts resolved Three files had conflicts due to parallel work on master: 1. **`CHANGELOG.md`** — Master added entries for issues #631 and #630 (TDD failing tests for session list/create DI bugs). Our #628 entry was placed ahead of those entries in the Unreleased section. 2. **`noxfile.py`** — Master independently added `--listener robot/tdd_expected_fail_listener.py` (as a string literal) to both `integration_tests` and `slow_integration_tests`. Our branch already had this via the `tdd_listener` variable (resolved absolute path). Resolved by keeping our variable-based approach and removing the duplicate from master. Also preserved our `--exclude tdd_fixture` flag that master didn't have. 3. **`robot/tdd_expected_fail_listener.py`** — Master had a simpler version (only `end_test`, no tag validation, no `start_test`, no logging, `Any` types). Our comprehensive version (full tag validation, proper Robot Framework types, logging, docstrings) replaces it entirely, as this is the issue that was meant to implement the full listener per the acceptance criteria. ### Quality gates (all pass on rebased branch) - `nox -s lint`: PASS - `nox -s typecheck`: PASS (0 errors, 1 warning) - `nox -s unit_tests`: PASS (10,431 scenarios, 370 features) - `nox -s integration_tests`: PASS (1,457 tests) - `nox -s coverage_report`: PASS (98% >= 97% threshold) PR !673 is now mergeable (`merge_base` matches current master HEAD).
Author
Owner

PM Acknowledgment — Day 31 (2026-03-11)

@hurui200320 — excellent work rebasing PR #673 and resolving all 3 conflicts. The PR is now clean and all quality gates pass.

Next step: PR #673 needs a reviewer. @CoreRasurae — would you be available to review this? Given that you're implementing the Behave counterpart (#627/PR #665), cross-reviewing makes sense for consistency.

This is a critical path item — the TDD infrastructure (#627 + #628) unblocks the entire TDD pipeline.

## PM Acknowledgment — Day 31 (2026-03-11) @hurui200320 — excellent work rebasing PR #673 and resolving all 3 conflicts. The PR is now clean and all quality gates pass. **Next step:** PR #673 needs a reviewer. @CoreRasurae — would you be available to review this? Given that you're implementing the Behave counterpart (#627/PR #665), cross-reviewing makes sense for consistency. This is a critical path item — the TDD infrastructure (#627 + #628) unblocks the entire TDD pipeline.
Member

Implementation Note — Review Round 4 (2026-03-16)

Rebase and Review Response

Rebased feature/m5-robot-tdd-tags onto latest master (3b6b1d2). After rebase, PR #673 contains only the single commit for #628 — previous scope-bundling concern (C-1) from Hamza's review is resolved as those commits are now on master.

Additional code changes in this round

  1. Extracted shared mock helper (features/mocks/tdd_test_helpers.py): The _make_mock_scenario() function was duplicated between robot/helper_tdd_tag_validation.py and features/steps/tdd_tag_validation_steps.py. Moved to a shared location per M-5 finding. Both files now import make_mock_scenario from the shared module.

  2. XML message attribute fallback (M-7): Added status_el.get("message", "") check between text and <msg> element fallback in both _run_fixture() and _run_multi_fixture(). RF 7.3+ may use either placement.

  3. New tdd_bug_alone edge case fixture: Added robot/fixtures/tdd_bug_alone.robot + cmd_tdd_bug_alone_valid() + test case to cover the missing edge case where tdd_bug is used alone without tdd_bug_<N> or tdd_expected_fail.

  4. Documentation improvements: Fixed "longname" → "full_name" in listener docstring (L-6), added comment documenting @ prefix divergence (L-7), and added design note explaining str | None return type choice in _validate_tdd_tags (nit).

Quality gates

All pass: lint (0 violations), typecheck (0 errors), unit_tests (10,815 scenarios), integration_tests (1,519 tests, including Robot.Tdd Tag Validation passing in 55.7s), coverage_report (97% >= 97%).

Key locations

  • Shared mock: features.mocks.tdd_test_helpers.make_mock_scenario (commit c7bdb62b)
  • New fixture: robot/fixtures/tdd_bug_alone.robot (commit c7bdb62b)
  • New command: robot/helper_tdd_tag_validation.cmd_tdd_bug_alone_valid (commit c7bdb62b)
  • XML fallback: robot/helper_tdd_tag_validation._run_fixture and _run_multi_fixture (commit c7bdb62b)
## Implementation Note — Review Round 4 (2026-03-16) ### Rebase and Review Response Rebased `feature/m5-robot-tdd-tags` onto latest master (`3b6b1d2`). After rebase, PR #673 contains only the single commit for #628 — previous scope-bundling concern (C-1) from Hamza's review is resolved as those commits are now on master. ### Additional code changes in this round 1. **Extracted shared mock helper** (`features/mocks/tdd_test_helpers.py`): The `_make_mock_scenario()` function was duplicated between `robot/helper_tdd_tag_validation.py` and `features/steps/tdd_tag_validation_steps.py`. Moved to a shared location per M-5 finding. Both files now import `make_mock_scenario` from the shared module. 2. **XML message attribute fallback** (M-7): Added `status_el.get("message", "")` check between text and `<msg>` element fallback in both `_run_fixture()` and `_run_multi_fixture()`. RF 7.3+ may use either placement. 3. **New `tdd_bug_alone` edge case fixture**: Added `robot/fixtures/tdd_bug_alone.robot` + `cmd_tdd_bug_alone_valid()` + test case to cover the missing edge case where `tdd_bug` is used alone without `tdd_bug_<N>` or `tdd_expected_fail`. 4. **Documentation improvements**: Fixed "longname" → "full_name" in listener docstring (L-6), added comment documenting `@` prefix divergence (L-7), and added design note explaining `str | None` return type choice in `_validate_tdd_tags` (nit). ### Quality gates All pass: lint (0 violations), typecheck (0 errors), unit_tests (10,815 scenarios), integration_tests (1,519 tests, including `Robot.Tdd Tag Validation` passing in 55.7s), coverage_report (97% >= 97%). ### Key locations - Shared mock: `features.mocks.tdd_test_helpers.make_mock_scenario` (commit `c7bdb62b`) - New fixture: `robot/fixtures/tdd_bug_alone.robot` (commit `c7bdb62b`) - New command: `robot/helper_tdd_tag_validation.cmd_tdd_bug_alone_valid` (commit `c7bdb62b`) - XML fallback: `robot/helper_tdd_tag_validation._run_fixture` and `_run_multi_fixture` (commit `c7bdb62b`)
Author
Owner

PM Status — Day 36 (2026-03-16)

@hurui200320 — Thank you for the Round 4 rebase update posted today. Good to see the scope-bundling concern (C-1) from Hamza's review is now resolved via rebase.

Status: PR #673 has been rebased onto latest master (3b6b1d2), single commit for #628 only. This is on the critical path — feeds into the CI quality gate (#629).

Reviewer needed: @CoreRasurae — the Day 31 request for your review is still outstanding. Given that the PR is now clean (single commit, no scope bundling), please prioritize this review. Target: Day 37 EOD.

Who Action Deadline
@CoreRasurae Review PR #673 Day 37 EOD
@hurui200320 Address any review findings Day 38 EOD
## PM Status — Day 36 (2026-03-16) @hurui200320 — Thank you for the Round 4 rebase update posted today. Good to see the scope-bundling concern (C-1) from Hamza's review is now resolved via rebase. **Status**: PR #673 has been rebased onto latest master (`3b6b1d2`), single commit for #628 only. This is on the critical path — feeds into the CI quality gate (#629). **Reviewer needed**: @CoreRasurae — the Day 31 request for your review is still outstanding. Given that the PR is now clean (single commit, no scope bundling), please prioritize this review. Target: **Day 37 EOD**. | Who | Action | Deadline | |-----|--------|----------| | @CoreRasurae | Review PR #673 | Day 37 EOD | | @hurui200320 | Address any review findings | Day 38 EOD |
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Blocks
Depends on
Reference
cleveragents/cleveragents-core#628
No description provided.