test: add TDD bug-capture test for #932 — plan apply missing --yes flag #958
Merged
brent.edwards
merged 1 commits from 2026-03-21 00:13:18 +00:00
tdd/m4-plan-apply-yes-flag into master
Dismiss Review
Are you sure you want to dismiss this review?
Labels
Clear labels
auto/needs-reevaluation
controller-managed
overdue
auto/blocked-by-deps
auto/ci-timeout
auto/claimed-implementer
auto/claimed-merge
auto/claimed-reviewer
auto/driver-down
auto/invariant-violation
auto/last-attempt-tier-0
auto/last-attempt-tier-1
auto/last-attempt-tier-2
auto/last-attempt-tier-min
Automation Tracking
auto/needs-conflict-resolution
auto/needs-implementer
auto/postmortem
auto/ready-to-merge
auto/restart-throttled
auto/revert
auto/sentinel
auto/stale-inactivity
auto/unstable
Blocked
Needs Feedback
Signed-off: Owner
Signed-off: Scrum Master
Signed-off: Tech Lead
Spike
Controller deferred this PR; awaiting Phase 6+ scope-evaluator or operator re-enablement.
Auto-agents controller manages this PR/issue (see tools/controller/deploy/RUNBOOK.md). Remove this label to abandon controller management.
PR blocked by an open issue dependency. Operator must close the dep (or remove the dependency link) before the merge driver can act. Auto-cleared by merge_drive when no open deps remain.
Most recent merge cycle hit CI timeout. Driver excludes this PR while last merge_cycle row is < 30 min old; label persists thereafter as visible history.
Currently being processed by an implementer worker.
Currently being processed by the merge driver.
Currently being processed by a reviewer worker.
Merge driver heartbeat stale; pipeline halted. Closed automatically on next clean tick.
Detected master commit violating the strict merge invariant. Tracked as an issue (not a PR label); kept here for label completeness.
In-cycle escalation: most recent attempt ran at the Tier 0 slot (`tier-0`). Slot's model defined in .opencode/models/tiers.yaml.
In-cycle escalation: most recent attempt ran at the Tier 1 slot (`tier-1`). Slot's model defined in .opencode/models/tiers.yaml.
In-cycle escalation: most recent attempt ran at the Tier 2 slot (`tier-2`). Slot's model defined in .opencode/models/tiers.yaml. Gated behind IMPLEMENTER_ESCALATION_TIER2_ENABLED.
In-cycle escalation: most recent attempt ran at the Tier -1 slot (`tier-min`). Slot's model defined in .opencode/models/tiers.yaml. Suffix is ``-min`` (not ``--1``) so the Forgejo UI reads naturally.
Tracking issues used by the AI Automation system for agents to communicate and report.
Rebase conflict needs LLM conflict-resolver.
Failing CI needs implementer attention.
Documenting a driver incident or rollback.
Reviewer has APPROVED this PR and no later REQUEST_CHANGES is outstanding. The merge driver requires this label to even consider a PR for merging. Set by the reviewer worker on APPROVE; cleared on REQUEST_CHANGES.
Train repeatedly lost master-tempo races. Driver excludes via merge_cycle until cooldown elapses; label persists as visible history.
Revert PR backing out an invariant violation. Fast-tracked through the merge driver.
Sentinel PR duplicated from upstream into a personal fork by tools/duplicate_prs_to_fork.py for pipeline testing. Lives only in the fork; the canonical pipeline never sees it.
No implementer activity for N days. Flagged for human review. Auto-cleared on next push to head branch.
Repeatedly fails on current master (>= 3 ci-fail-on-rebased-sha releases in 12 h). Excluded from driver until human triage.
A ticket in a blocked state and unable to complete until some other task is completed first.
Bounty
$100
A bounty of $100 for any open-source contributor who provides a MR that solves this issue
Bounty
$1000
A bounty of $1000 for any open-source contributor who provides a MR that solves this issue
Bounty
$10000
A bounty of $10000 for any open-source contributor who provides a MR that solves this issue
Bounty
$20
A bounty of $20 for any open-source contributor who provides a MR that solves this issue
Bounty
$2000
A bounty of $2000 for any open-source contributor who provides a MR that solves this issue
Bounty
$250
A bounty of $250 for any open-source contributor who provides a MR that solves this issue
Bounty
$50
A bounty of $50 for any open-source contributor who provides a MR that solves this issue
Bounty
$500
A bounty of $500 for any open-source contributor who provides a MR that solves this issue
Bounty
$5000
A bounty of $5000 for any open-source contributor who provides a MR that solves this issue
Bounty
$750
A bounty of $750 for any open-source contributor who provides a MR that solves this issue
MoSCoW
Could have
Could have feature in order to satisfy the epic/legendary.
MoSCoW
Must have
Must have feature in order to satisfy the epic/legendary.
MoSCoW
Should have
Should have feature in order to satisfy the epic/legendary.
There are questions in the ticket that can not be completed until the project owner provides clarity.
Points
1
1 man-hours worth of work for an expert with no learning curve.
Points
13
13 man-hours worth of work for an expert with no learning curve.
Points
2
2 man-hours worth of work for an expert with no learning curve.
Points
21
21 man-hours worth of work for an expert with no learning curve.
Points
3
3 man-hours worth of work for an expert with no learning curve.
Points
34
34 man-hours worth of work for an expert with no learning curve.
Points
5
5 man-hours worth of work for an expert with no learning curve.
Points
55
55 man-hours worth of work for an expert with no learning curve.
Points
8
8 man-hours worth of work for an expert with no learning curve.
Points
88
88 man-hours worth of work for an expert with no learning curve.
Priority
Backlog
This ticket has backlogged priority and is not to be worked on yet
Priority
CI Blocker
Critical priority issue that blocks CI/CD pipeline and prevents PR merges
Priority
Critical
The priority is critical
Priority
High
The priority is high
Priority
Low
The priority is low
Priority
Medium
The priority is medium
When an epic or legendary is in review it must be signed off by owner, tech lead, and scrum master before being marked as completed.
When an epic or legendary is in review it must be signed off by owner, tech lead, and scrum master before being marked as completed.
When an epic or legendary is in review it must be signed off by owner, tech lead, and scrum master before being marked as completed.
A ticket for learning a tool or technology that is needed to be able to do future planning and design.
State
Completed
The ticket has been fully implemented, completed, and merged with the source code. This label should only be applied once a ticket is closed.
State
Duplicate
A ticket that represents the same content as an existing ticket.
State
In Progress
A ticket that is actively being developed.
State
In Review
A ticket that has had some code completed to implement but is waiting to pass peer review and is not yet merged in.
State
Paused
This ticket's work started but wasn't finished. It's on hold (likely in a feature branch) and will be resumed later, either due to a blocker or a delay.
State
Unverified
All new tickets start in this state. A developer may set it to show the ticket is unverified. This means we haven't agreed to work on it. It will either move to a verified state or be closed as wontdo.
State
Verified
The issue has been verified by a developer as legitimate. It will be worked on and verified tickets are now considered part of the backlog.
State
Wont Do
This ticket has been decided it wont be done. This may mean the bug has been determined to not be real (cant verify) or the feature is one we have decided we dont want to adopt.
Type
Automation
Any edits or discussion about the AI automated coding system.
Type
Bug
Something that doesnt work as intended.
Type
Discussion
Anytime a ticket represents a discussion about a subject and doesnt fall into one of the other categories.
Type
Documentation
An error or improvement needed in the documentation.
Type
Epic
Any first tier epic. That is, an epic which contains only issues as children and will not have sub-epics.
Type
Feature
Some new functionality not present.
Type
Legendary
A type of Epic which will contain other Epics.
Type
Refactor
A code change that restructures existing code without changing its external behavior.
Type
Support
Someone needs help using the project.
Type
Task
A generic task that doesnt fit into the other type categories.
Type
Testing
Work exclusively focusing on fixing or expanding testing.
Projects
Clear projects
No project
Assignees
aditya (Aditya Chhabra)
aleenaumair (Aleena Umair)
brent.edwards (Brent Edwards)
CoreRasurae (Luis Mendes)
drew (Drew Morris)
eugen.thaci (Eugen Thaci)
freemo (Jeffrey Phillips Freeman)
HAL9000 (HAL 9000)
HAL9001 (HAL9001)
hamza.khyari (Hamza Khyari)
hurui200320 (Rui Hu)
justin.morris
khird (Kyle Hird)
org.cleveragents
Clear assignees
No Assignees
brent.edwards
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: cleveragents/cleveragents-core#958
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Delete Branch "tdd/m4-plan-apply-yes-flag"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
TDD expected-fail tests proving bug #932 exists: the
plan apply(lifecycle-apply) command does not accept the--yes/-yflag required by the specification. The flag should skip the confirmation prompt before applying plan changes.Tests Added
Behave scenarios (
features/tdd_plan_apply_yes_flag.feature):lifecycle-apply --yesshould be accepted (tests--yeslong flag)lifecycle-apply -yshould be accepted (tests-yshort flag)Tags:
@tdd_expected_fail @tdd_bug @tdd_bug_932Robot Framework tests (
robot/tdd_plan_apply_yes_flag.robot):check-yes-long— invokes CLI with--yes, asserts no "No such option" errorcheck-yes-short— invokes CLI with-y, asserts no "No such option" errorHow the Bug Is Proven
The
lifecycle_apply_planfunction (cleveragents.cli.commands.plan) defines onlyplan_idand--formatparameters — no--yes/-y. When tests invokelifecycle-apply --yes, Typer/Click rejects it with"No such option: --yes"and exit code 2. The assertion that this error is absent fails, confirming bug #932. The@tdd_expected_failtag inverts this to a pass.Quality Gates
nox -s lintnox -s typechecknox -s unit_testsnox -s integration_testsnox -s coverage_reportCloses #950
b52a3ad3c7to2975a875b4PM Status — Day 36 (2026-03-16)
TDD bug-capture test for #932 (plan apply missing
--yesflag).Priority: Critical path — this TDD test must pass before the fix PR for #932 can proceed.
Status check: @brent.edwards — this PR was submitted Day 35 with 0 comments. Does it need a rebase? Please confirm it is ready for review.
Reviewer assignment: @hamza.khyari — please fast-track review this TDD PR. Target: review complete by Day 37 EOD. This is blocking a Priority/Critical bug fix.
PM Review — TDD PR for Bug #932 (Plan Apply --yes Flag)
PR #958: TDD bug-capture test for #932 — plan apply missing --yes flag
Review Summary
This is a well-structured TDD PR that correctly captures bug #932. Approving.
TDD PR Verification Checklist
tdd/prefix (tdd/m4-plan-apply-yes-flag)@tdd_bug,@tdd_bug_932,@tdd_expected_fail--yesand-yflags) capture the exact bugtest:prefix conventionAssessment
lifecycle_apply_planlacks the--yes/-yparameter, which causes Typer/Click to reject the flag with exit code 2. The@tdd_expected_failinversion makes this pass CI.--yes/-yis actually accepted by the command.tdd/m4-plan-apply-yes-flagformat matching issue #950's Metadata.Minor Note
The PR is currently mergeable but the earlier PM comment noted it may need a rebase. @brent.edwards — if the merge base has shifted, please rebase before merge.
Verdict: APPROVED. Merge when a second reviewer also approves (per CONTRIBUTING.md 2-reviewer rule). Suggest @hamza.khyari or @CoreRasurae for second review.
Status Update — Day 37
@freemo — Confirmed: PR #958 is ready for review. Master has been merged in today (previously unmergeable due to robot timeout conflicts — resolved by keeping master's
timeout=120s on_timeout=killvalues).Branch
tdd/m4-plan-apply-yes-flagis now mergeable and up to date with master. No upstream dependencies — this is a standalone TDD test PR.Ready for @hamza.khyari's fast-track review.
PM Day 37 — Fast-Track Review Follow-up
This TDD PR has been open since Day 35 and has 1 APPROVED review. It is mergeable and the author confirmed it is rebased (Day 37). The TDD verification checklist passes:
tdd/m4-plan-apply-yes-flag✅@tdd_bug,@tdd_bug_932,@tdd_expected_fail✅This PR is blocking the fix for bug #932 (Priority/Critical). @hamza.khyari — your fast-track review was requested Day 36 with a Day 37 EOD deadline. Please complete your review today. If you cannot review by EOD, @CoreRasurae has been added as a backup reviewer.
The TDD test correctly captures that
lifecycle-applylacks the--yes/-yflag, which Typer rejects with exit code 2. The@tdd_expected_failtag inverts this failure to a CI pass.PM review follow-up — Day 37
1a69de68dfto2975a875b4New commits pushed, approval review dismissed automatically according to repository settings
Rebase Confirmed — Addressing @freemo PM Note
Branch
tdd/m4-plan-apply-yes-flagis rebased on currentorigin/master. Force-pushed to confirm clean state. Single commit, no merge commits, all gates passing.Ready for second reviewer (@hamza.khyari or @CoreRasurae as suggested by PM).
Code Review Report — PR #958 (TDD Bug #932: plan apply missing --yes flag)
Scope: All code changes on branch
tdd/m4-plan-apply-yes-flagvsorigin/master, including surrounding context incleveragents/cli/commands/plan.py,docs/specification.md, and existing TDD test patterns.Review cycles performed: 5 full global sweeps across all categories (bugs, logic errors, edge cases, security, test coverage, test flaws, performance, structure, spec compliance, conventions).
Files reviewed:
features/tdd_plan_apply_yes_flag.feature(22 lines, new)features/steps/tdd_plan_apply_yes_flag_steps.py(87 lines, new)robot/tdd_plan_apply_yes_flag.robot(33 lines, new)robot/helper_tdd_plan_apply_yes_flag.py(93 lines, new)No Critical or High Severity Issues Found
The implementation correctly captures bug #932, follows the project's established TDD conventions, and satisfies the acceptance criteria in issue #950.
Medium Severity
1. Behave assertions could be more resilient to Click/Typer wording changes
Category: Test Robustness
Files:
features/steps/tdd_plan_apply_yes_flag_steps.py:65-72,features/steps/tdd_plan_apply_yes_flag_steps.py:80-87The
thensteps only check"No such option" not in outputbut do not verifyexit_code != 2. Click/Typer exit code 2 specifically means "usage error" (unrecognised option). If a future Click/Typer version changes the error message wording (e.g., to "Unknown option: --yes"), the text-based assertion would falsely pass even though the bug is still present, causing the@tdd_expected_failinversion to report an unexpected pass.Suggestion: Add
assert context.result.exit_code != 2alongside the existing output check, or replace the text check with the exit code check entirely. Example:Low Severity
2. Robot helper only checks output text, not CliRunner exit code
Category: Test Robustness
Files:
robot/helper_tdd_plan_apply_yes_flag.py:49-55,robot/helper_tdd_plan_apply_yes_flag.py:63-69Same pattern as finding #1:
check_yes_long()andcheck_yes_short()check"No such option" in result.outputbut don't checkresult.exit_code == 2. This is lower impact than the Behave version because the Robot test chain also validates the helper's own exit code (rc=0) and sentinel string presence, providing a double-check layer.Suggestion: Optionally add
result.exit_code == 2as an additional condition in theifblock:3. Robot suite may need database isolation when bug is fixed
Category: Forward Compatibility
File:
robot/tdd_plan_apply_yes_flag.robot:10The suite uses
Setup Test EnvironmentwithoutSetup Database Isolation. Percommon.resourcedocumentation, suites that run helper scripts viaRun Processshould also callSetup Database Isolationso that concurrent pabot workers don't contend on the same SQLite file.Currently this is not a problem because the test's CliRunner invocation fails at flag parsing (exit code 2) before any database access occurs. However, when the bug is fixed and the
@tdd_expected_failtag is removed, the command will proceed past flag parsing and likely attempt database operations withDUMMY_PLAN_ID, which could produce unexpected errors in a concurrent pabot environment.This will need to be addressed as part of the fix for #932, not in this TDD PR, so it is informational for now.
Verification Summary
@tdd_expected_fail @tdd_bug @tdd_bug_932present--yesplan correctline 2577,plan rollbackline 2893)lifecycle-apply)@tdd_expected_failinversion logic correctagents plan apply [--yes|-y] <PLAN_ID>)2975a875b4to981786a362Review Fixes Applied — Addressing @CoreRasurae Findings
Commit:
981786a3(force-pushed)Fixes Applied
exit_code != 2check alongside text check in bothstep_assert_yes_recognisedandstep_assert_y_recognised. Now assertsexit_code != 2 and "No such option" not in output.result.exit_code == 2as additional condition in bothcheck_yes_long()andcheck_yes_short()— now fails on either text match or exit code 2.Setup Database Isolationwhen removing@tdd_expected_fail.All changes are backwards-compatible — the assertions remain correct under the current bug state (exit code 2 + "No such option" in output → assertion fails →
@tdd_expected_failinverts to pass).981786a362to2c14dafbc1Code Review — PR #958
Verdict: Approve with comments (1 Major, 1 Minor)
Scope: 8 files, +294/−55 (three-dot diff). 4 new files (TDD tests), 4 modified files (timeout cleanup). Single commit. Lint PASS, Typecheck PASS.
TDD Test Logic: Correct
The core TDD test logic is sound:
lifecycle-applycurrently has no--yes/-yflag (plan.py:1803-1817— onlyplan_idand--format)lifecycle-apply --yes DUMMY_PLAN_ID, which Click/Typer rejects with exit code 2 and"No such option: --yes"exit_code != 2 and "No such option" not in outputcorrectly fails (proving the bug)@tdd_expected_failinverts the failure to a CI pass--yestolifecycle-apply), the assertion passes and removing@tdd_expected_failmakes the test pass normally--yes(long) and-y(short) variants are covered@tdd_expected_fail @tdd_bug @tdd_bug_932on both Behave feature and Robot test casesMajor
PROCESS-1: Out-of-scope timeout/on_timeout cleanup in 4 unrelated Robot files
robot/cli_core.robot,robot/core_cli_commands.robot,robot/database_integration.robot,robot/scientific_paper_e2e_test.robotThe diff modifies 4 Robot files unrelated to bug #932 — changing
timeout=120stotimeout=60sand removingon_timeout=killacross ~50 lines. The commit body acknowledges these as "pre-existing test failures" fixes, but per CONTRIBUTING.md single-commit/single-responsibility rules, unrelated cleanup should be a separate issue and commit.The changes themselves are harmless —
Run Processdefaultson_timeouttoterminate(SIGTERM with 30s escalation to SIGKILL), which is actually more graceful thankill(immediate SIGKILL). The timeout reductions (120s→60s for fast commands likeversion/diagnostics) are reasonable.Recommendation: Split these into a separate
chore:issue/commit, or at minimum document in the commit body which specific test failures these fixes resolved.Minor
PROCESS-2: No CHANGELOG entry
No entry for #950 in the Unreleased section. Per CONTRIBUTING.md, PRs should include a changelog update. For TDD test-only PRs this is debatable — a brief entry like "Added TDD bug-capture tests for #932 (plan apply missing --yes flag). (#950)" would suffice.
Phase Summary
Round 2 Follow-Up — 1 new nit
Deep second pass verified:
AmbiguousSteprisk.@tdd_expected_failworks correctly:apply_tdd_inversion()(environment.py:123-225) usesscenario.effective_tagswhich propagates feature-level tags to all child scenarios. Same pattern astdd_checkpoint_real_rollback.featureandtdd_subplan_spawn_orchestration.feature.environment.py:172-185) checks whether failed steps raisedAssertionErrorspecifically — infrastructure failures are not masked.rc=0assertion fails → listener inverts to pass.Nit
CODE-3: Behave steps use generic
context.runner/context.resultinstead of namespaced attributesfeatures/steps/tdd_plan_apply_yes_flag_steps.py:29,42,49Other step files namespace their context attributes to prevent cross-contamination (
context.lcc_result,context.r2cov_result,context.init_yes_result, etc.). This file uses barecontext.runnerandcontext.result, which overlap withplan_lifecycle_cli_steps.py:88-92. No current collision since all steps in each scenario come from the same file, but a maintenance risk if scenarios are later mixed with steps from other files.Recommendation: Rename to
context.apply_yes_runner/context.apply_yes_resultto follow the codebase convention.Updated totals (Round 1 + Round 2):
check comments
Response to @hamza.khyari's Review — PR #958
Thank you for the thorough review, Hamza. Great catches on all three findings. Here's my plan:
PROCESS-1 (Major): Out-of-scope timeout/on_timeout cleanup in 4 Robot files
Agreed — these should be in a separate commit. I'll revert the timeout changes from these 4 unrelated files and move them to a dedicated
chore:commit (or separate PR if preferred). The TDD commit should contain only the 4 new test files.PROCESS-2 (Minor): No CHANGELOG entry
Good point. I'll add a brief entry:
"Added TDD bug-capture tests for #932 (plan apply missing --yes flag). (#950)".CODE-3 (Nit): Generic
context.runner/context.resultnamingAgreed — I'll rename to
context.apply_yes_runner/context.apply_yes_resultto follow the codebase convention and prevent potential cross-contamination.Also confirming that @CoreRasurae's findings (#1 and #2 — exit code checks) were already applied in commit
981786a3.Working on all fixes now.
2c14dafbc1tof0162ef637f0162ef637tofcbf02c5bfReview Findings — Verified & Resolved
All three review findings are addressed in the rebased branch:
CHANGELOG.mdline 11:- Added TDD bug-capture tests for #932 (plan apply missing --yes flag). (#950)context.runner→context.apply_yes_runner,context.result→context.apply_yes_resultthroughoutfeatures/steps/tdd_plan_apply_yes_flag_steps.pyLint/typecheck results:
nox -s lint— All checks passednox -s typecheck— 0 errors, 1 pre-existing warning (unrelated file)Final commit:
fcbf02c5(single commit on master)Review Fixes Verified — Commit
fcbf02c5Branch rebased onto
origin/master(ad98d41d). All 3 review findings resolved:## Unreleasedcontext.apply_yes_runner/context.apply_yes_resultQuality Gates
nox -s lint— PASSnox -s typecheck— PASS (0 errors)ad98d41dCode Review — PR #958
test: TDD bug-capture test for #932 — plan apply missing --yes flagClean, well-scoped TDD test PR. The 2 Behave scenarios + 2 Robot tests verify that
lifecycle-applyrecognizes the--yes/-yflag (exit code != 2, no "No such option" text). TDD tags@tdd_expected_fail @tdd_bug @tdd_bug_932are correctly applied.Approved with one note:
mergeable: false). The code itself is clean — once rebased against currentmaster, this should be ready to merge.Planning Agent — Day 39 PR Status
This TDD PR (#958) for bug #932 is ready for merge. All review findings have been addressed:
context.apply_yes_runner/context.apply_yes_result.@hamza.khyari provided a thorough review and @CoreRasurae provided early findings. @brent.edwards addressed all items. The TDD test logic is correct — verifies that
lifecycle-apply --yescurrently fails with exit code 2 ("No such option"), which@tdd_expected_failinverts to a CI pass.Merge recommendation: Ready for merge. Reviewers assigned: @freemo and @hamza.khyari. Two approvals needed per CONTRIBUTING.md. Once merged, @hurui200320 (Rui) can begin the bugfix PR for #932.
fcbf02c5bfto53d3f82228New commits pushed, approval review dismissed automatically according to repository settings
53d3f82228to28ffd64832Fixed
nox -s benchmark_regressioncrash caused by missingASV_BASE_SHAenvironment variable. Updatednoxfile.pyto default tomasterif the variable is unset.28ffd64832tob48e17bb09b48e17bb09to7c0d42f92a