- Changed gate logic to pass trivially when no TDD test exists for a referenced issue
- This allows feature PRs (like #629) to pass without requiring a TDD test
- Only enforces tag removal IF a TDD test exists for the bug
- Extracted helper functions from tdd_quality_gate_steps.py to tdd_quality_gate_helpers.py
- Reduced steps file from 541 lines to 472 lines (under 500-line limit)
- All quality gates passing: lint, typecheck
Fixes#629
Add an automated quality gate that enforces TDD bug fix workflow rules
on pull requests. The gate parses PR descriptions for bug-closing
keywords (Closes/Fixes/Resolves #N, ISSUES CLOSED: #N), searches the
codebase for corresponding TDD tests tagged @tdd_bug_N, and verifies
that @tdd_expected_fail tags have been removed.
Key components:
- scripts/tdd_quality_gate.py: Main quality gate script with PR
description parsing, TDD test discovery, and tag removal verification.
All public functions validate arguments fail-fast and are statically
typed.
- noxfile.py: New tdd_quality_gate session that reads PR_DESCRIPTION
from the environment and runs the quality gate script.
- .forgejo/workflows/ci.yml: New tdd_quality_gate CI job that runs
only on pull_request events, passing the PR body as PR_DESCRIPTION.
- features/tdd_quality_gate.feature: 46 Behave scenarios covering PR
parsing, TDD test search, tag removal verification, full gate logic,
robot diff handling, edge cases, argument validation, bool guards,
co-located bug false-positive guard, and main() CLI entry point.
- features/steps/tdd_quality_gate_steps.py: Step definitions for all
Behave scenarios using temporary directories for isolation.
- robot/tdd_quality_gate.robot: 15 Robot Framework integration tests
exercising the gate end-to-end via a helper subprocess.
- robot/helper_tdd_quality_gate.py: Helper script for Robot tests with
sentinel-based sub-commands.
Review-round fixes applied:
- check_expected_fail_removed now uses _contains_tag_token for
word-boundary matching (avoids false positives on partial tag names)
- Diff expected-fail removal detection tracks flags at file level
instead of per-hunk (fixes false negatives when tags span hunks)
- parse_bug_refs filters out issue number zero
- Redundant double error reporting eliminated (file-level check
short-circuits the diff-level check)
- run_quality_gate returns (errors, bug_refs) tuple to avoid
redundant re-parsing in main()
- Regex compilation cached via functools.lru_cache
- Nox session no longer installs the full project (stdlib only)
- CI checkout uses fetch-depth: 0 for reliable merge-base resolution
Review-round 2 fixes applied:
- _diff_has_expected_fail_removal_for_bug now requires the removed
line to contain both the expected-fail tag and the specific bug tag
(fixes false positives when two bugs share the same test file)
- check_expected_fail_removed error messages use the correct tag
prefix per file type (@tdd_bug_N for .feature, tdd_bug_N for .robot)
- bool values rejected by bug-number validation guards in
find_tdd_tests, check_expected_fail_removed, and
_diff_has_expected_fail_removal_for_bug
- File-read error handling catches UnicodeDecodeError alongside OSError
(root-safe unreadable-file handling via invalid-UTF-8 test fixture)
- Temp directory cleanup added to after_scenario hook in environment.py
- 8 new Behave scenarios: bool type guards (2), co-located bug
false-positive regression (1), run_quality_gate argument validation
(3), and main() CLI entry point exit codes (2)
Review-round 3 fixes applied:
- Synthetic PR diff helper (_default_pr_diff_for_bug_refs) now
auto-detects .robot vs .feature file type from the temp search
tree and generates the matching diff format (fixes under-tested
robot-format diff code path in multi-bug integration scenarios)
- check_expected_fail_removed test step now filters files by bug
tag via find_tdd_tests before checking (matches production path
in run_quality_gate)
- after_scenario temp directory cleanup no longer sets
context.temp_dir = None (fixes cleanup conflict with
cli_init_yes_flag_steps.py cleanup functions that run after hooks)
- 2 new Behave scenarios: multi-line PR description parsing, and
non-string pr_diff type guard for run_quality_gate
ISSUES CLOSED: #629
Created scripts/create_template_db.py that builds a pre-migrated SQLite
template database using Base.metadata.create_all() + alembic stamp
(~5ms for 34 tables, vs ~0.5-3s x 25 Alembic migrations per scenario).
Nox unit_tests and coverage_report sessions generate the template before
test execution and propagate CLEVERAGENTS_TEMPLATE_DB env var to all
workers.
features/environment.py before_all() installs a monkey-patch on
MigrationRunner.init_or_upgrade that copies the template for fresh
scenario temp DBs, falling through to real migrations for :memory:,
existing files, and migration-runner unit tests.
Quick wins: sleep(0.5) -> sleep(0.05) in cli_streaming wait step;
removed redundant Background re-declaration in cli_streaming.feature
scenario 7.
ISSUES CLOSED: #483
HAL9000
CoreRasurae
freemo
brent.edwards
