test: add TDD bug-capture test for #932 — plan apply missing --yes flag #958
No reviewers
Labels
No labels
auto/needs-reevaluation
controller-managed
auto/blocked-by-deps
auto/ci-timeout
auto/claimed-implementer
auto/claimed-merge
auto/claimed-reviewer
auto/driver-down
auto/invariant-violation
auto/last-attempt-tier-0
auto/last-attempt-tier-1
auto/last-attempt-tier-2
auto/last-attempt-tier-min
Automation Tracking
auto/needs-conflict-resolution
auto/needs-implementer
auto/postmortem
auto/ready-to-merge
auto/restart-throttled
auto/revert
auto/sentinel
auto/stale-inactivity
auto/unstable
Blocked
Bounty
$100
Bounty
$1000
Bounty
$10000
Bounty
$20
Bounty
$2000
Bounty
$250
Bounty
$50
Bounty
$500
Bounty
$5000
Bounty
$750
MoSCoW
Could have
MoSCoW
Must have
MoSCoW
Should have
Needs Feedback
Points
1
Points
13
Points
2
Points
21
Points
3
Points
34
Points
5
Points
55
Points
8
Points
88
Priority
Backlog
Priority
CI Blocker
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Signed-off: Owner
Signed-off: Scrum Master
Signed-off: Tech Lead
Spike
State
Completed
State
Duplicate
State
In Progress
State
In Review
State
Paused
State
Unverified
State
Verified
State
Wont Do
Type
Automation
Type
Bug
Type
Discussion
Type
Documentation
Type
Epic
Type
Feature
Type
Legendary
Type
Refactor
Type
Support
Type
Task
Type
Testing
No project
No assignees
4 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
cleveragents/cleveragents-core!958
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "tdd/m4-plan-apply-yes-flag"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
TDD expected-fail tests proving bug #932 exists: the
plan apply(lifecycle-apply) command does not accept the--yes/-yflag required by the specification. The flag should skip the confirmation prompt before applying plan changes.Tests Added
Behave scenarios (
features/tdd_plan_apply_yes_flag.feature):lifecycle-apply --yesshould be accepted (tests--yeslong flag)lifecycle-apply -yshould be accepted (tests-yshort flag)Tags:
@tdd_expected_fail @tdd_bug @tdd_bug_932Robot Framework tests (
robot/tdd_plan_apply_yes_flag.robot):check-yes-long— invokes CLI with--yes, asserts no "No such option" errorcheck-yes-short— invokes CLI with-y, asserts no "No such option" errorHow the Bug Is Proven
The
lifecycle_apply_planfunction (cleveragents.cli.commands.plan) defines onlyplan_idand--formatparameters — no--yes/-y. When tests invokelifecycle-apply --yes, Typer/Click rejects it with"No such option: --yes"and exit code 2. The assertion that this error is absent fails, confirming bug #932. The@tdd_expected_failtag inverts this to a pass.Quality Gates
nox -s lintnox -s typechecknox -s unit_testsnox -s integration_testsnox -s coverage_reportCloses #950
b52a3ad3c72975a875b4PM Status — Day 36 (2026-03-16)
TDD bug-capture test for #932 (plan apply missing
--yesflag).Priority: Critical path — this TDD test must pass before the fix PR for #932 can proceed.
Status check: @brent.edwards — this PR was submitted Day 35 with 0 comments. Does it need a rebase? Please confirm it is ready for review.
Reviewer assignment: @hamza.khyari — please fast-track review this TDD PR. Target: review complete by Day 37 EOD. This is blocking a Priority/Critical bug fix.
PM Review — TDD PR for Bug #932 (Plan Apply --yes Flag)
PR #958: TDD bug-capture test for #932 — plan apply missing --yes flag
Review Summary
This is a well-structured TDD PR that correctly captures bug #932. Approving.
TDD PR Verification Checklist
tdd/prefix (tdd/m4-plan-apply-yes-flag)@tdd_bug,@tdd_bug_932,@tdd_expected_fail--yesand-yflags) capture the exact bugtest:prefix conventionAssessment
lifecycle_apply_planlacks the--yes/-yparameter, which causes Typer/Click to reject the flag with exit code 2. The@tdd_expected_failinversion makes this pass CI.--yes/-yis actually accepted by the command.tdd/m4-plan-apply-yes-flagformat matching issue #950's Metadata.Minor Note
The PR is currently mergeable but the earlier PM comment noted it may need a rebase. @brent.edwards — if the merge base has shifted, please rebase before merge.
Verdict: APPROVED. Merge when a second reviewer also approves (per CONTRIBUTING.md 2-reviewer rule). Suggest @hamza.khyari or @CoreRasurae for second review.
Status Update — Day 37
@freemo — Confirmed: PR #958 is ready for review. Master has been merged in today (previously unmergeable due to robot timeout conflicts — resolved by keeping master's
timeout=120s on_timeout=killvalues).Branch
tdd/m4-plan-apply-yes-flagis now mergeable and up to date with master. No upstream dependencies — this is a standalone TDD test PR.Ready for @hamza.khyari's fast-track review.
PM Day 37 — Fast-Track Review Follow-up
This TDD PR has been open since Day 35 and has 1 APPROVED review. It is mergeable and the author confirmed it is rebased (Day 37). The TDD verification checklist passes:
tdd/m4-plan-apply-yes-flag✅@tdd_bug,@tdd_bug_932,@tdd_expected_fail✅This PR is blocking the fix for bug #932 (Priority/Critical). @hamza.khyari — your fast-track review was requested Day 36 with a Day 37 EOD deadline. Please complete your review today. If you cannot review by EOD, @CoreRasurae has been added as a backup reviewer.
The TDD test correctly captures that
lifecycle-applylacks the--yes/-yflag, which Typer rejects with exit code 2. The@tdd_expected_failtag inverts this failure to a CI pass.PM review follow-up — Day 37
1a69de68df2975a875b4New commits pushed, approval review dismissed automatically according to repository settings
Rebase Confirmed — Addressing @freemo PM Note
Branch
tdd/m4-plan-apply-yes-flagis rebased on currentorigin/master. Force-pushed to confirm clean state. Single commit, no merge commits, all gates passing.Ready for second reviewer (@hamza.khyari or @CoreRasurae as suggested by PM).
Code Review Report — PR #958 (TDD Bug #932: plan apply missing --yes flag)
Scope: All code changes on branch
tdd/m4-plan-apply-yes-flagvsorigin/master, including surrounding context incleveragents/cli/commands/plan.py,docs/specification.md, and existing TDD test patterns.Review cycles performed: 5 full global sweeps across all categories (bugs, logic errors, edge cases, security, test coverage, test flaws, performance, structure, spec compliance, conventions).
Files reviewed:
features/tdd_plan_apply_yes_flag.feature(22 lines, new)features/steps/tdd_plan_apply_yes_flag_steps.py(87 lines, new)robot/tdd_plan_apply_yes_flag.robot(33 lines, new)robot/helper_tdd_plan_apply_yes_flag.py(93 lines, new)No Critical or High Severity Issues Found
The implementation correctly captures bug #932, follows the project's established TDD conventions, and satisfies the acceptance criteria in issue #950.
Medium Severity
1. Behave assertions could be more resilient to Click/Typer wording changes
Category: Test Robustness
Files:
features/steps/tdd_plan_apply_yes_flag_steps.py:65-72,features/steps/tdd_plan_apply_yes_flag_steps.py:80-87The
thensteps only check"No such option" not in outputbut do not verifyexit_code != 2. Click/Typer exit code 2 specifically means "usage error" (unrecognised option). If a future Click/Typer version changes the error message wording (e.g., to "Unknown option: --yes"), the text-based assertion would falsely pass even though the bug is still present, causing the@tdd_expected_failinversion to report an unexpected pass.Suggestion: Add
assert context.result.exit_code != 2alongside the existing output check, or replace the text check with the exit code check entirely. Example:Low Severity
2. Robot helper only checks output text, not CliRunner exit code
Category: Test Robustness
Files:
robot/helper_tdd_plan_apply_yes_flag.py:49-55,robot/helper_tdd_plan_apply_yes_flag.py:63-69Same pattern as finding #1:
check_yes_long()andcheck_yes_short()check"No such option" in result.outputbut don't checkresult.exit_code == 2. This is lower impact than the Behave version because the Robot test chain also validates the helper's own exit code (rc=0) and sentinel string presence, providing a double-check layer.Suggestion: Optionally add
result.exit_code == 2as an additional condition in theifblock:3. Robot suite may need database isolation when bug is fixed
Category: Forward Compatibility
File:
robot/tdd_plan_apply_yes_flag.robot:10The suite uses
Setup Test EnvironmentwithoutSetup Database Isolation. Percommon.resourcedocumentation, suites that run helper scripts viaRun Processshould also callSetup Database Isolationso that concurrent pabot workers don't contend on the same SQLite file.Currently this is not a problem because the test's CliRunner invocation fails at flag parsing (exit code 2) before any database access occurs. However, when the bug is fixed and the
@tdd_expected_failtag is removed, the command will proceed past flag parsing and likely attempt database operations withDUMMY_PLAN_ID, which could produce unexpected errors in a concurrent pabot environment.This will need to be addressed as part of the fix for #932, not in this TDD PR, so it is informational for now.
Verification Summary
@tdd_expected_fail @tdd_bug @tdd_bug_932present--yesplan correctline 2577,plan rollbackline 2893)lifecycle-apply)@tdd_expected_failinversion logic correctagents plan apply [--yes|-y] <PLAN_ID>)2975a875b4981786a362Review Fixes Applied — Addressing @CoreRasurae Findings
Commit:
981786a3(force-pushed)Fixes Applied
exit_code != 2check alongside text check in bothstep_assert_yes_recognisedandstep_assert_y_recognised. Now assertsexit_code != 2 and "No such option" not in output.result.exit_code == 2as additional condition in bothcheck_yes_long()andcheck_yes_short()— now fails on either text match or exit code 2.Setup Database Isolationwhen removing@tdd_expected_fail.All changes are backwards-compatible — the assertions remain correct under the current bug state (exit code 2 + "No such option" in output → assertion fails →
@tdd_expected_failinverts to pass).981786a3622c14dafbc1Code Review — PR #958
Verdict: Approve with comments (1 Major, 1 Minor)
Scope: 8 files, +294/−55 (three-dot diff). 4 new files (TDD tests), 4 modified files (timeout cleanup). Single commit. Lint PASS, Typecheck PASS.
TDD Test Logic: Correct
The core TDD test logic is sound:
lifecycle-applycurrently has no--yes/-yflag (plan.py:1803-1817— onlyplan_idand--format)lifecycle-apply --yes DUMMY_PLAN_ID, which Click/Typer rejects with exit code 2 and"No such option: --yes"exit_code != 2 and "No such option" not in outputcorrectly fails (proving the bug)@tdd_expected_failinverts the failure to a CI pass--yestolifecycle-apply), the assertion passes and removing@tdd_expected_failmakes the test pass normally--yes(long) and-y(short) variants are covered@tdd_expected_fail @tdd_bug @tdd_bug_932on both Behave feature and Robot test casesMajor
PROCESS-1: Out-of-scope timeout/on_timeout cleanup in 4 unrelated Robot files
robot/cli_core.robot,robot/core_cli_commands.robot,robot/database_integration.robot,robot/scientific_paper_e2e_test.robotThe diff modifies 4 Robot files unrelated to bug #932 — changing
timeout=120stotimeout=60sand removingon_timeout=killacross ~50 lines. The commit body acknowledges these as "pre-existing test failures" fixes, but per CONTRIBUTING.md single-commit/single-responsibility rules, unrelated cleanup should be a separate issue and commit.The changes themselves are harmless —
Run Processdefaultson_timeouttoterminate(SIGTERM with 30s escalation to SIGKILL), which is actually more graceful thankill(immediate SIGKILL). The timeout reductions (120s→60s for fast commands likeversion/diagnostics) are reasonable.Recommendation: Split these into a separate
chore:issue/commit, or at minimum document in the commit body which specific test failures these fixes resolved.Minor
PROCESS-2: No CHANGELOG entry
No entry for #950 in the Unreleased section. Per CONTRIBUTING.md, PRs should include a changelog update. For TDD test-only PRs this is debatable — a brief entry like "Added TDD bug-capture tests for #932 (plan apply missing --yes flag). (#950)" would suffice.
Phase Summary
Round 2 Follow-Up — 1 new nit
Deep second pass verified:
AmbiguousSteprisk.@tdd_expected_failworks correctly:apply_tdd_inversion()(environment.py:123-225) usesscenario.effective_tagswhich propagates feature-level tags to all child scenarios. Same pattern astdd_checkpoint_real_rollback.featureandtdd_subplan_spawn_orchestration.feature.environment.py:172-185) checks whether failed steps raisedAssertionErrorspecifically — infrastructure failures are not masked.rc=0assertion fails → listener inverts to pass.Nit
CODE-3: Behave steps use generic
context.runner/context.resultinstead of namespaced attributesfeatures/steps/tdd_plan_apply_yes_flag_steps.py:29,42,49Other step files namespace their context attributes to prevent cross-contamination (
context.lcc_result,context.r2cov_result,context.init_yes_result, etc.). This file uses barecontext.runnerandcontext.result, which overlap withplan_lifecycle_cli_steps.py:88-92. No current collision since all steps in each scenario come from the same file, but a maintenance risk if scenarios are later mixed with steps from other files.Recommendation: Rename to
context.apply_yes_runner/context.apply_yes_resultto follow the codebase convention.Updated totals (Round 1 + Round 2):
check comments
Response to @hamza.khyari's Review — PR #958
Thank you for the thorough review, Hamza. Great catches on all three findings. Here's my plan:
PROCESS-1 (Major): Out-of-scope timeout/on_timeout cleanup in 4 Robot files
Agreed — these should be in a separate commit. I'll revert the timeout changes from these 4 unrelated files and move them to a dedicated
chore:commit (or separate PR if preferred). The TDD commit should contain only the 4 new test files.PROCESS-2 (Minor): No CHANGELOG entry
Good point. I'll add a brief entry:
"Added TDD bug-capture tests for #932 (plan apply missing --yes flag). (#950)".CODE-3 (Nit): Generic
context.runner/context.resultnamingAgreed — I'll rename to
context.apply_yes_runner/context.apply_yes_resultto follow the codebase convention and prevent potential cross-contamination.Also confirming that @CoreRasurae's findings (#1 and #2 — exit code checks) were already applied in commit
981786a3.Working on all fixes now.
2c14dafbc1f0162ef637f0162ef637fcbf02c5bfReview Findings — Verified & Resolved
All three review findings are addressed in the rebased branch:
CHANGELOG.mdline 11:- Added TDD bug-capture tests for #932 (plan apply missing --yes flag). (#950)context.runner→context.apply_yes_runner,context.result→context.apply_yes_resultthroughoutfeatures/steps/tdd_plan_apply_yes_flag_steps.pyLint/typecheck results:
nox -s lint— All checks passednox -s typecheck— 0 errors, 1 pre-existing warning (unrelated file)Final commit:
fcbf02c5(single commit on master)Review Fixes Verified — Commit
fcbf02c5Branch rebased onto
origin/master(ad98d41d). All 3 review findings resolved:## Unreleasedcontext.apply_yes_runner/context.apply_yes_resultQuality Gates
nox -s lint— PASSnox -s typecheck— PASS (0 errors)ad98d41dCode Review — PR #958
test: TDD bug-capture test for #932 — plan apply missing --yes flagClean, well-scoped TDD test PR. The 2 Behave scenarios + 2 Robot tests verify that
lifecycle-applyrecognizes the--yes/-yflag (exit code != 2, no "No such option" text). TDD tags@tdd_expected_fail @tdd_bug @tdd_bug_932are correctly applied.Approved with one note:
mergeable: false). The code itself is clean — once rebased against currentmaster, this should be ready to merge.Planning Agent — Day 39 PR Status
This TDD PR (#958) for bug #932 is ready for merge. All review findings have been addressed:
context.apply_yes_runner/context.apply_yes_result.@hamza.khyari provided a thorough review and @CoreRasurae provided early findings. @brent.edwards addressed all items. The TDD test logic is correct — verifies that
lifecycle-apply --yescurrently fails with exit code 2 ("No such option"), which@tdd_expected_failinverts to a CI pass.Merge recommendation: Ready for merge. Reviewers assigned: @freemo and @hamza.khyari. Two approvals needed per CONTRIBUTING.md. Once merged, @hurui200320 (Rui) can begin the bugfix PR for #932.
fcbf02c5bf53d3f82228New commits pushed, approval review dismissed automatically according to repository settings
53d3f8222828ffd64832Fixed
nox -s benchmark_regressioncrash caused by missingASV_BASE_SHAenvironment variable. Updatednoxfile.pyto default tomasterif the variable is unset.28ffd64832b48e17bb09b48e17bb097c0d42f92a