From a39d04d8f91346cca25cd136b1e65103b7ee5048 Mon Sep 17 00:00:00 2001 From: CleverThis Date: Wed, 13 May 2026 00:04:56 +0000 Subject: [PATCH] feat(agents): add BDD tests for implementation-pool-supervisor PR compliance checklist Extend BDD test coverage to verify all 8 mandatory PR compliance checklist items (CHANGELOG.md update, CONTRIBUTORS.md update, commit footer, CI verification, BDD/Behave tests, Epic reference, label application via forgejo-label-manager, and milestone assignment) are present in the implementation-pool-supervisor.md agent definition. New feature file and step definitions ensure reproducible compliance verification alongside existing implementation-supervisor.BDD tests. Parent Epic: #9779 ISSUES CLOSED: #11052 --- CHANGELOG.md | 11 +- CONTRIBUTORS.md | 7 + ...mpliance_checklist_pool_supervisor.feature | 58 ++++++ ...pliance_checklist_pool_supervisor_steps.py | 186 ++++++++++++++++++ 4 files changed, 261 insertions(+), 1 deletion(-) create mode 100644 features/pr_compliance_checklist_pool_supervisor.feature create mode 100644 features/steps/pr_compliance_checklist_pool_supervisor_steps.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 5cf3abfa7..8066855b3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -30,7 +30,16 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). "ValueError") with no diagnostic detail, making production debugging impossible. The handler now includes the error message text and full traceback in the structlog warning entry. Removed `@tdd_expected_fail` tag - from the TDD test so both scenarios run as normal regression guards. (#988) + from the TDD test so both scenarios run as normal regression guards. (#988) + +- **BDD tests added for implementation-pool-supervisor PR compliance checklist** + (#11052): Extended BDD test coverage to verify all 8 mandatory PR compliance + checklist items (CHANGELOG.md update, CONTRIBUTORS.md update, commit footer, + CI verification, BDD/Behave tests, Epic reference, label application via + forgejo-label-manager, and milestone assignment) are present in the + `implementation-pool-supervisor.md` agent definition. New feature file + `features/pr_compliance_checklist_pool_supervisor.feature` with dedicated step + definitions ensures reproducible compliance verification. ### Added diff --git a/CONTRIBUTORS.md b/CONTRIBUTORS.md index 735c33416..e44c05c95 100644 --- a/CONTRIBUTORS.md +++ b/CONTRIBUTORS.md @@ -43,3 +43,10 @@ Below are some of the specific details of various contributions. * HAL 9000 has contributed database resource types (PostgreSQL, SQLite) with transaction-based sandbox strategy: implemented ``DatabaseResourceHandler`` providing full CRUD operations (`read`, `write`, `delete`, `list_children`) and connection validation with automatic credential masking for PostgreSQL and SQLite backends. Includes ``TransactionSandbox`` infrastructure wired into ``SandboxFactory``, BDD test coverage in ``features/database_resources.feature``, and Robot Framework integration tests in ``robot/database_resources.robot`` (PR #10591 / issue #8608, Epic #8568). * HAL 9000 has contributed the agents plan rollback command (PR #8674 / issue #8557): implemented checkpoint-based plan state restoration with the `agents plan rollback []` CLI command as part of Epic #8493, enabling plans to be restored to previous checkpoints, discarding post-checkpoint decisions, and resuming execution from the rolled-back state. Supported by `--yes/-y`, `--to-checkpoint`, and `--format/-f` flags. Includes comprehensive BDD test coverage (>= 97%) for rollback, decision discarding, and plan resume functionality. * HAL 9000 has contributed the PyYAML security upgrade (PR #11012 / issue #9055): added `pyyaml>=6.0.3` dependency constraint to address known YAML parsing vulnerabilities. + +* HAL 9000 has contributed mandatory PR compliance checklist BDD tests for + implementation-pool-supervisor (#11052): created new feature file + `pr_compliance_checklist_pool_supervisor.feature` and step definitions file + covering all 8 mandatory checklist items (CHANGELOG.md, CONTRIBUTORS.md, commit + footer, CI passes, BDD tests, Epic reference, labels, milestone) to ensure + reproducible compliance verification for the implementation pool supervisor agent. diff --git a/features/pr_compliance_checklist_pool_supervisor.feature b/features/pr_compliance_checklist_pool_supervisor.feature new file mode 100644 index 000000000..6fd452a08 --- /dev/null +++ b/features/pr_compliance_checklist_pool_supervisor.feature @@ -0,0 +1,58 @@ +@mock_only +Feature: PR Compliance Checklist in Implementation Pool Supervisor + + As an implementation pool supervisor + I want to pass a mandatory PR compliance checklist to every worker prompt + So that workers complete all required items before creating a PR and avoid systemic merge blockers + + Background: + Given the implementation-pool-supervisor.md agent definition exists + + Scenario: Pool supervisor contains PR Compliance Checklist section + When I read the implementation pool supervisor agent definition + Then the agent file includes the PR compliance checklist section + And the checklist is marked as MANDATORY + + Scenario: Pool supervisor item 1 — CHANGELOG.md update required + When I read the implementation pool supervisor agent definition + Then the agent file includes a CHANGELOG.md checklist item + And the item instructs workers to add an entry under the Unreleased section + + Scenario: Pool supervisor item 2 — CONTRIBUTORS.md update required + When I read the implementation pool supervisor agent definition + Then the agent file includes a CONTRIBUTORS.md checklist item + And the item instructs workers to add or update their contribution entry + + Scenario: Pool supervisor item 3 — Commit footer required + When I read the implementation pool supervisor agent definition + Then the agent file includes a commit footer checklist item + And the item specifies the ISSUES CLOSED footer format + + Scenario: Pool supervisor item 4 — CI must pass before PR creation + When I read the implementation pool supervisor agent definition + Then the agent file includes a CI passes checklist item + And the item instructs workers to verify all quality gates are green + + Scenario: Pool supervisor item 5 — BDD/Behave tests required + When I read the implementation pool supervisor agent definition + Then the agent file includes a BDD tests checklist item + And the item instructs workers to add or update Behave feature files + + Scenario: Pool supervisor item 6 — Epic reference required in PR description + When I read the implementation pool supervisor agent definition + Then the agent file includes an Epic reference / parent Epic checklist item + And the item instructs workers to reference the parent Epic issue number + + Scenario: Pool supervisor item 7 — Labels must be applied + When I read the implementation pool supervisor agent definition + Then the agent file includes a labels checklist item + And the item instructs workers to apply labels via forgejo-label-manager + + Scenario: Pool supervisor item 8 — Milestone must be assigned + When I read the implementation pool supervisor agent definition + Then the agent file includes a milestone checklist item + And the item instructs workers to assign the earliest open milestone + + Scenario: All 8 checklist items are present in the pool supervisor + When I read the implementation pool supervisor agent definition + Then the agent file contains all 8 mandatory checklist items diff --git a/features/steps/pr_compliance_checklist_pool_supervisor_steps.py b/features/steps/pr_compliance_checklist_pool_supervisor_steps.py new file mode 100644 index 000000000..c4bb9c47d --- /dev/null +++ b/features/steps/pr_compliance_checklist_pool_supervisor_steps.py @@ -0,0 +1,186 @@ +"""Step definitions for PR compliance checklist in implementation pool supervisor.""" + +from pathlib import Path +from typing import Any + +from behave import given, then, when + +PROJECT_ROOT = Path(__file__).resolve().parents[2] +AGENT_DEF_PATH = PROJECT_ROOT / ".opencode" / "agents" / "implementation-pool-supervisor.md" + + +@given("the implementation-pool-supervisor.md agent definition exists") +def step_agent_def_exists(context: Any) -> None: + """Verify the implementation pool supervisor agent definition file exists.""" + assert AGENT_DEF_PATH.exists(), f"Agent definition not found at {AGENT_DEF_PATH}" + context.agent_def_path = AGENT_DEF_PATH + + +@when("I read the implementation pool supervisor agent definition") +def step_read_agent_def(context: Any) -> None: + """Read the implementation pool supervisor agent definition.""" + context.agent_def_content = AGENT_DEF_PATH.read_text(encoding="utf-8") + + +@then("the agent file includes the PR compliance checklist section") +def step_prompt_includes_checklist(context: Any) -> None: + """Verify the agent file includes the PR compliance checklist.""" + assert "PR Compliance Checklist" in context.agent_def_content, ( + "Agent file does not include 'PR Compliance Checklist'" + ) + + +@then("the checklist is marked as MANDATORY") +def step_checklist_is_mandatory(context: Any) -> None: + """Verify the checklist is marked as MANDATORY.""" + assert "MANDATORY" in context.agent_def_content, ( + "PR Compliance Checklist is not marked as MANDATORY" + ) + + +@then("the agent file includes a CHANGELOG.md checklist item") +def step_prompt_includes_changelog_item(context: Any) -> None: + """Verify the agent file includes a CHANGELOG.md checklist item.""" + assert "CHANGELOG.md" in context.agent_def_content, ( + "Agent file does not include a CHANGELOG.md checklist item" + ) + + +@then("the item instructs workers to add an entry under the Unreleased section") +def step_changelog_item_unreleased(context: Any) -> None: + """Verify the CHANGELOG.md item mentions the Unreleased section.""" + assert "[Unreleased]" in context.agent_def_content, ( + "CHANGELOG.md checklist item does not mention the [Unreleased] section" + ) + + +@then("the agent file includes a CONTRIBUTORS.md checklist item") +def step_prompt_includes_contributors_item(context: Any) -> None: + """Verify the agent file includes a CONTRIBUTORS.md checklist item.""" + assert "CONTRIBUTORS.md" in context.agent_def_content, ( + "Agent file does not include a CONTRIBUTORS.md checklist item" + ) + + +@then("the item instructs workers to add or update their contribution entry") +def step_contributors_item_add_update(context: Any) -> None: + """Verify the CONTRIBUTORS.md item instructs workers to add or update.""" + assert "add or update" in context.agent_def_content, ( + "CONTRIBUTORS.md checklist item does not instruct workers to add or update" + ) + + +@then("the agent file includes a commit footer checklist item") +def step_prompt_includes_commit_footer_item(context: Any) -> None: + """Verify the agent file includes a commit footer checklist item.""" + assert "Commit footer" in context.agent_def_content, ( + "Agent file does not include a commit footer checklist item" + ) + + +@then("the item specifies the ISSUES CLOSED footer format") +def step_commit_footer_issues_closed(context: Any) -> None: + """Verify the commit footer item specifies the ISSUES CLOSED format.""" + assert "ISSUES CLOSED" in context.agent_def_content, ( + "Commit footer checklist item does not specify the ISSUES CLOSED format" + ) + + +@then("the agent file includes a CI passes checklist item") +def step_prompt_includes_ci_item(context: Any) -> None: + """Verify the agent file includes a CI passes checklist item.""" + assert "CI passes" in context.agent_def_content, ( + "Agent file does not include a CI passes checklist item" + ) + + +@then("the item instructs workers to verify all quality gates are green") +def step_ci_item_quality_gates(context: Any) -> None: + """Verify the CI item instructs workers to verify quality gates.""" + assert "quality gates" in context.agent_def_content, ( + "CI checklist item does not mention quality gates" + ) + + +@then("the agent file includes a BDD tests checklist item") +def step_prompt_includes_bdd_item(context: Any) -> None: + """Verify the agent file includes a BDD/Behave tests checklist item.""" + assert "BDD/Behave tests" in context.agent_def_content, ( + "Agent file does not include a BDD/Behave tests checklist item" + ) + + +@then("the item instructs workers to add or update Behave feature files") +def step_bdd_item_feature_files(context: Any) -> None: + """Verify the BDD item instructs workers to add or update feature files.""" + assert "added or updated" in context.agent_def_content, ( + "BDD checklist item does not instruct workers to add or update feature files" + ) + + +@then("the agent file includes an Epic reference / parent Epic checklist item") +def step_prompt_includes_epic_item(context: Any) -> None: + """Verify the agent file includes an Epic reference checklist item.""" + assert "Epic reference" in context.agent_def_content or \ + "parent Epic" in context.agent_def_content, ( + "Agent file does not include an Epic reference checklist item" + ) + + +@then("the item instructs workers to reference the parent Epic issue number") +def step_epic_item_parent_reference(context: Any) -> None: + """Verify the Epic item instructs workers to reference the parent Epic.""" + assert "parent Epic" in context.agent_def_content, ( + "Epic checklist item does not instruct workers to reference the parent Epic" + ) + + +@then("the agent file includes a labels checklist item") +def step_prompt_includes_labels_item(context: Any) -> None: + """Verify the agent file includes a labels checklist item.""" + assert "Labels" in context.agent_def_content, ( + "Agent file does not include a labels checklist item" + ) + + +@then("the item instructs workers to apply labels via forgejo-label-manager") +def step_labels_item_forgejo_label_manager(context: Any) -> None: + """Verify the labels item instructs workers to use forgejo-label-manager.""" + assert "forgejo-label-manager" in context.agent_def_content, ( + "Labels checklist item does not mention forgejo-label-manager" + ) + + +@then("the agent file includes a milestone checklist item") +def step_prompt_includes_milestone_item(context: Any) -> None: + """Verify the agent file includes a milestone checklist item.""" + assert "Milestone" in context.agent_def_content, ( + "Agent file does not include a milestone checklist item" + ) + + +@then("the item instructs workers to assign the earliest open milestone") +def step_milestone_item_earliest(context: Any) -> None: + """Verify the milestone item instructs workers to assign the earliest open milestone.""" + assert "earliest open milestone" in context.agent_def_content, ( + "Milestone checklist item does not mention the earliest open milestone" + ) + + +@then("the agent file contains all 8 mandatory checklist items") +def step_prompt_contains_all_8_items(context: Any) -> None: + """Verify the agent file contains all 8 mandatory checklist items.""" + required_items = [ + "CHANGELOG.md", + "CONTRIBUTORS.md", + "ISSUES CLOSED", + "CI passes", + "BDD/Behave tests", + "Epic reference", + "forgejo-label-manager", + "earliest open milestone", + ] + missing = [item for item in required_items if item not in context.agent_def_content] + assert not missing, ( + f"Agent file is missing the following checklist items: {missing}" + ) -- 2.52.0