TDD: Write failing test for #932 — plan apply missing --yes flag #950

Closed
opened 2026-03-14 04:09:00 +00:00 by freemo · 5 comments
Owner

Metadata

  • Commit Message: test: add TDD bug-capture test for #932 — plan apply missing --yes flag
  • Branch: tdd/m4-plan-apply-yes-flag

Background and Context

This is the TDD counterpart to bug #932. Per the project's Test-Driven Development
workflow for bugs (see CONTRIBUTING.md > Bug Fix Workflow), the first step in fixing any
bug is to write a test that captures the buggy behavior. The test is tagged with
@tdd_bug, @tdd_bug_932, and @tdd_expected_fail so that it passes CI while
the bug is still unfixed. Once the fix is implemented in #932, the
@tdd_expected_fail tag will be removed and the test will run normally.

See #932 for full bug details.

Bug summary: The plan apply command is missing the --yes flag that the specification
requires. The specification mandates a confirmation prompt before applying plan changes,
with --yes as the flag to skip the prompt. The current implementation lacks this flag
entirely, meaning there is no confirmation safety net before applying changes.

Expected Behavior

A new test exists that:

  1. Captures the exact failure described in #932 — invoking plan apply with --yes flag
    results in an error because the flag does not exist.
  2. Is tagged with @tdd_bug, @tdd_bug_932, and @tdd_expected_fail.
  3. Passes CI via the expected-failure mechanism (the underlying assertion fails, confirming
    the bug exists, but the tag inversion causes the test to pass).
  4. Would fail CI if the bug were fixed without removing the @tdd_expected_fail tag.

Acceptance Criteria

  • A test is written that captures the bug behavior described in #932.
  • The test is tagged with @tdd_bug, @tdd_bug_932, and @tdd_expected_fail.
  • The @tdd_expected_fail tag causes the test to pass CI (the underlying assertion
    fails as expected, proving the bug exists).
  • The test is specific enough that it will pass normally (without the tag) only when the
    bug is genuinely fixed.
  • Tag validation rules pass: @tdd_bug_932 has corresponding @tdd_bug, and
    @tdd_expected_fail has both.
  • A pull request is opened from tdd/m4-plan-apply-yes-flag to master, CI passes,
    and the PR is merged through the normal merge process.

Definition of Done

This issue is complete when:

  • All subtasks below are completed and checked off.
  • A Git commit is created where the first line of the commit message matches
    the Commit Message in Metadata exactly, followed by a blank line, then additional
    lines providing relevant details about the test and what bug behavior it captures.
  • The commit is pushed to the remote on the branch matching the Branch in
    Metadata exactly.
  • The commit is submitted as a pull request to master, reviewed, CI passes,
    and the PR is merged before this issue is marked done.

Subtasks

  • Code: Analyze bug #932 to identify the exact failure condition — the plan apply
    command does not accept a --yes flag, and there is no confirmation prompt.
  • Code: Determine the appropriate test type (Behave unit test, Robot integration test,
    or both) and file location for the reproducing test.
  • Tests (Behave): Write a Behave scenario in features/ that captures the bug. Tag
    the scenario with @tdd_bug, @tdd_bug_932, and @tdd_expected_fail. The
    scenario must invoke plan apply --yes and assert that the flag is recognized
    and the command proceeds without prompting. Name the scenario descriptively to
    indicate it is a bug regression test.
  • Tests (Robot): Add a Robot test in robot/ with equivalent tags that captures the
    failure at the integration level — invoke the CLI with --yes flag and verify
    it is accepted.
  • Docs: Add a comment in the test file explaining that this test was written to capture
    bug #932 and uses the @tdd_expected_fail tag until the fix is merged.
  • Quality: Verify CI passes with the tagged test. Confirm the underlying assertion
    fails for the correct reason (the bug itself, not a test setup error).
  • Quality: Verify tag validation rules pass (@tdd_bug_932 has @tdd_bug, and
    @tdd_expected_fail has both).
  • Quality: Verify coverage >=97% via nox -s coverage_report. If coverage is <97%
    then review the current unit test coverage report at build/coverage.xml and use it
    to write new Behave based unit tests to improve code coverage. Repeat until >=97%.
  • Quality: Run nox (all default sessions, including benchmark), fix any errors if
    needed ensuring nox passes across entire code base.
## Metadata - **Commit Message**: `test: add TDD bug-capture test for #932 — plan apply missing --yes flag` - **Branch**: `tdd/m4-plan-apply-yes-flag` ## Background and Context This is the TDD counterpart to bug #932. Per the project's Test-Driven Development workflow for bugs (see `CONTRIBUTING.md` > Bug Fix Workflow), the first step in fixing any bug is to write a test that captures the buggy behavior. The test is tagged with `@tdd_bug`, `@tdd_bug_932`, and `@tdd_expected_fail` so that it passes CI while the bug is still unfixed. Once the fix is implemented in #932, the `@tdd_expected_fail` tag will be removed and the test will run normally. See #932 for full bug details. **Bug summary**: The `plan apply` command is missing the `--yes` flag that the specification requires. The specification mandates a confirmation prompt before applying plan changes, with `--yes` as the flag to skip the prompt. The current implementation lacks this flag entirely, meaning there is no confirmation safety net before applying changes. ## Expected Behavior A new test exists that: 1. Captures the exact failure described in #932 — invoking `plan apply` with `--yes` flag results in an error because the flag does not exist. 2. Is tagged with `@tdd_bug`, `@tdd_bug_932`, and `@tdd_expected_fail`. 3. Passes CI via the expected-failure mechanism (the underlying assertion fails, confirming the bug exists, but the tag inversion causes the test to pass). 4. Would fail CI if the bug were fixed without removing the `@tdd_expected_fail` tag. ## Acceptance Criteria - [ ] A test is written that captures the bug behavior described in #932. - [ ] The test is tagged with `@tdd_bug`, `@tdd_bug_932`, and `@tdd_expected_fail`. - [ ] The `@tdd_expected_fail` tag causes the test to pass CI (the underlying assertion fails as expected, proving the bug exists). - [ ] The test is specific enough that it will pass normally (without the tag) only when the bug is genuinely fixed. - [ ] Tag validation rules pass: `@tdd_bug_932` has corresponding `@tdd_bug`, and `@tdd_expected_fail` has both. - [ ] A pull request is opened from `tdd/m4-plan-apply-yes-flag` to `master`, CI passes, and the PR is merged through the normal merge process. ## Definition of Done This issue is complete when: - All subtasks below are completed and checked off. - A Git commit is created where the **first line** of the commit message matches the Commit Message in Metadata exactly, followed by a blank line, then additional lines providing relevant details about the test and what bug behavior it captures. - The commit is pushed to the remote on the branch matching the **Branch** in Metadata exactly. - The commit is submitted as a **pull request** to `master`, reviewed, CI passes, and the PR is **merged** before this issue is marked done. ## Subtasks - [ ] Code: Analyze bug #932 to identify the exact failure condition — the `plan apply` command does not accept a `--yes` flag, and there is no confirmation prompt. - [ ] Code: Determine the appropriate test type (Behave unit test, Robot integration test, or both) and file location for the reproducing test. - [ ] Tests (Behave): Write a Behave scenario in `features/` that captures the bug. Tag the scenario with `@tdd_bug`, `@tdd_bug_932`, and `@tdd_expected_fail`. The scenario must invoke `plan apply --yes` and assert that the flag is recognized and the command proceeds without prompting. Name the scenario descriptively to indicate it is a bug regression test. - [ ] Tests (Robot): Add a Robot test in `robot/` with equivalent tags that captures the failure at the integration level — invoke the CLI with `--yes` flag and verify it is accepted. - [ ] Docs: Add a comment in the test file explaining that this test was written to capture bug #932 and uses the `@tdd_expected_fail` tag until the fix is merged. - [ ] Quality: Verify CI passes with the tagged test. Confirm the underlying assertion fails for the correct reason (the bug itself, not a test setup error). - [ ] Quality: Verify tag validation rules pass (`@tdd_bug_932` has `@tdd_bug`, and `@tdd_expected_fail` has both). - [ ] Quality: Verify coverage >=97% via `nox -s coverage_report`. If coverage is <97% then review the current unit test coverage report at `build/coverage.xml` and use it to write new Behave based unit tests to improve code coverage. Repeat until >=97%. - [ ] Quality: Run `nox` (all default sessions, including benchmark), fix any errors if needed ensuring nox passes across **entire** code base.
freemo added this to the v3.3.0 milestone 2026-03-14 04:12:28 +00:00
Author
Owner

Dependency (TDD workflow): This TDD test issue blocks bug fix #932. This issue must be completed and merged first so the @tdd_expected_fail test exists before the fix is implemented.

Blocks: #932

**Dependency (TDD workflow):** This TDD test issue blocks bug fix #932. This issue must be completed and merged first so the `@tdd_expected_fail` test exists before the fix is implemented. **Blocks:** #932
Member

Implemented in PR #958 (commit b52a3ad3).

Created 4 files with TDD expected-fail tests proving bug #932 exists:

  • features/tdd_plan_apply_yes_flag.feature (21 lines)
  • features/steps/tdd_plan_apply_yes_flag_steps.py (85 lines)
  • robot/tdd_plan_apply_yes_flag.robot (33 lines)
  • robot/helper_tdd_plan_apply_yes_flag.py (88 lines)

All quality gates pass: lint, typecheck, unit_tests (10808 scenarios), integration_tests (1508 tests), coverage_report (98%).

Implemented in PR #958 (commit `b52a3ad3`). Created 4 files with TDD expected-fail tests proving bug #932 exists: - `features/tdd_plan_apply_yes_flag.feature` (21 lines) - `features/steps/tdd_plan_apply_yes_flag_steps.py` (85 lines) - `robot/tdd_plan_apply_yes_flag.robot` (33 lines) - `robot/helper_tdd_plan_apply_yes_flag.py` (88 lines) All quality gates pass: lint, typecheck, unit_tests (10808 scenarios), integration_tests (1508 tests), coverage_report (98%).
Member

Implementation Notes

Design Decisions

Test approach: Both Behave (unit) and Robot (integration) tests verify the same thing — that invoking lifecycle-apply --yes and lifecycle-apply -y triggers a "No such option" error from Typer/Click, proving the flag doesn't exist.

Command name: Used lifecycle-apply (the current name) rather than apply (the spec name), since lifecycle-apply is what's actually registered. Issue #881 tracks the rename.

Two scenarios per framework: One for --yes (long flag) and one for -y (short flag), since both must work per specification.

Bug Mechanism

The lifecycle_apply_plan function at cleveragents.cli.commands.plan defines only plan_id: str and format: str parameters. No yes: bool = typer.Option(False, "--yes", "-y") parameter exists. When Typer/Click receives --yes, it rejects with exit code 2 and "No such option: --yes". The test asserts this error is absent, which fails, confirming the bug.

Files Created

File Lines Purpose
features/tdd_plan_apply_yes_flag.feature 21 2 Behave scenarios (--yes and -y)
features/steps/tdd_plan_apply_yes_flag_steps.py 85 Step definitions using CliRunner
robot/tdd_plan_apply_yes_flag.robot 33 2 Robot test cases with on_timeout=kill
robot/helper_tdd_plan_apply_yes_flag.py 88 Python helper with 2 subcommands

Quality Gates

Session Result
nox -s lint PASS
nox -s typecheck PASS (0 errors)
nox -s unit_tests PASS (10,808 scenarios)
nox -s integration_tests PASS (1,508 tests)
nox -s coverage_report 98% (>= 97%)

Commit

2975a875 on branch tdd/m4-plan-apply-yes-flag (single squashed commit)

PR

PR #958

## Implementation Notes ### Design Decisions **Test approach**: Both Behave (unit) and Robot (integration) tests verify the same thing — that invoking `lifecycle-apply --yes` and `lifecycle-apply -y` triggers a "No such option" error from Typer/Click, proving the flag doesn't exist. **Command name**: Used `lifecycle-apply` (the current name) rather than `apply` (the spec name), since `lifecycle-apply` is what's actually registered. Issue #881 tracks the rename. **Two scenarios per framework**: One for `--yes` (long flag) and one for `-y` (short flag), since both must work per specification. ### Bug Mechanism The `lifecycle_apply_plan` function at `cleveragents.cli.commands.plan` defines only `plan_id: str` and `format: str` parameters. No `yes: bool = typer.Option(False, "--yes", "-y")` parameter exists. When Typer/Click receives `--yes`, it rejects with exit code 2 and "No such option: --yes". The test asserts this error is absent, which fails, confirming the bug. ### Files Created | File | Lines | Purpose | |---|---|---| | `features/tdd_plan_apply_yes_flag.feature` | 21 | 2 Behave scenarios (--yes and -y) | | `features/steps/tdd_plan_apply_yes_flag_steps.py` | 85 | Step definitions using CliRunner | | `robot/tdd_plan_apply_yes_flag.robot` | 33 | 2 Robot test cases with on_timeout=kill | | `robot/helper_tdd_plan_apply_yes_flag.py` | 88 | Python helper with 2 subcommands | ### Quality Gates | Session | Result | |---|---| | `nox -s lint` | PASS | | `nox -s typecheck` | PASS (0 errors) | | `nox -s unit_tests` | PASS (10,808 scenarios) | | `nox -s integration_tests` | PASS (1,508 tests) | | `nox -s coverage_report` | 98% (>= 97%) | ### Commit `2975a875` on branch `tdd/m4-plan-apply-yes-flag` (single squashed commit) ### PR [PR #958](https://git.cleverthis.com/cleveragents/cleveragents-core/pulls/958)
Author
Owner

PM Acknowledgment — Day 36

@brent.edwards — Good implementation. The TDD tests for bug #932 are well-structured:

  • 4 files created covering both Behave and Robot frameworks
  • Two scenarios per framework (long --yes and short -y flags)
  • Correct command name (lifecycle-apply) per current implementation
  • Bug mechanism clearly documented (Typer/Click rejects unknown option)
  • All quality gates passing

PR #958 is in review but not mergeable — needs rebase. Please prioritize the rebase so this can proceed through peer review.

Current priority ordering for @brent.edwards:

  1. Rebase PR #958 (TDD for #932, Critical)
  2. Support PR #929 review (TDD for #822, Critical)
  3. Rebase PR #962 (estimation actor, v3.5.0)
  4. Address review findings on PRs #787, #791, #798
  5. Start TDD #840 (context tier bug)

PM acknowledgment — Day 36

## PM Acknowledgment — Day 36 @brent.edwards — Good implementation. The TDD tests for bug #932 are well-structured: - 4 files created covering both Behave and Robot frameworks - Two scenarios per framework (long `--yes` and short `-y` flags) - Correct command name (`lifecycle-apply`) per current implementation - Bug mechanism clearly documented (Typer/Click rejects unknown option) - All quality gates passing PR #958 is in review but **not mergeable** — needs rebase. Please prioritize the rebase so this can proceed through peer review. **Current priority ordering for @brent.edwards:** 1. Rebase PR #958 (TDD for #932, Critical) 2. Support PR #929 review (TDD for #822, Critical) 3. Rebase PR #962 (estimation actor, v3.5.0) 4. Address review findings on PRs #787, #791, #798 5. Start TDD #840 (context tier bug) --- *PM acknowledgment — Day 36*
Member

Acknowledgment — PM Day 36

@freemo — Acknowledged. PR #958 rebase is on my queue. Will rebase onto current master as part of the next batch of branch updates. Priority ordering noted:

  1. PR #958 rebase (this — Critical, blocks #932)
  2. PR #929 review support (Critical, blocks #822)
  3. PR #962 rebase (v3.5.0, lower priority)
  4. Remaining integration test review fixes
## Acknowledgment — PM Day 36 @freemo — Acknowledged. PR #958 rebase is on my queue. Will rebase onto current master as part of the next batch of branch updates. Priority ordering noted: 1. PR #958 rebase (this — Critical, blocks #932) 2. PR #929 review support (Critical, blocks #822) 3. PR #962 rebase (v3.5.0, lower priority) 4. Remaining integration test review fixes
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
cleveragents/cleveragents-core#950
No description provided.