feat(agents): add mandatory PR compliance checklist BDD tests for implementation-pool-supervisor #11181

Closed
HAL9000 wants to merge 1 commits from feature/pr-compliance-pool-supervisor into master
4 changed files with 261 additions and 1 deletions
+10 -1
View File
@@ -30,7 +30,16 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
"ValueError") with no diagnostic detail, making production debugging
impossible. The handler now includes the error message text and full
traceback in the structlog warning entry. Removed `@tdd_expected_fail` tag
from the TDD test so both scenarios run as normal regression guards. (#988)
from the TDD test so both scenarios run as normal regression guards. (#988)
- **BDD tests added for implementation-pool-supervisor PR compliance checklist**
(#11052): Extended BDD test coverage to verify all 8 mandatory PR compliance
checklist items (CHANGELOG.md update, CONTRIBUTORS.md update, commit footer,
CI verification, BDD/Behave tests, Epic reference, label application via
forgejo-label-manager, and milestone assignment) are present in the
`implementation-pool-supervisor.md` agent definition. New feature file
`features/pr_compliance_checklist_pool_supervisor.feature` with dedicated step
definitions ensures reproducible compliance verification.
### Added
+7
View File
@@ -43,3 +43,10 @@ Below are some of the specific details of various contributions.
* HAL 9000 has contributed database resource types (PostgreSQL, SQLite) with transaction-based sandbox strategy: implemented ``DatabaseResourceHandler`` providing full CRUD operations (`read`, `write`, `delete`, `list_children`) and connection validation with automatic credential masking for PostgreSQL and SQLite backends. Includes ``TransactionSandbox`` infrastructure wired into ``SandboxFactory``, BDD test coverage in ``features/database_resources.feature``, and Robot Framework integration tests in ``robot/database_resources.robot`` (PR #10591 / issue #8608, Epic #8568).
* HAL 9000 has contributed the agents plan rollback command (PR #8674 / issue #8557): implemented checkpoint-based plan state restoration with the `agents plan rollback <plan-id> [<checkpoint-id>]` CLI command as part of Epic #8493, enabling plans to be restored to previous checkpoints, discarding post-checkpoint decisions, and resuming execution from the rolled-back state. Supported by `--yes/-y`, `--to-checkpoint`, and `--format/-f` flags. Includes comprehensive BDD test coverage (>= 97%) for rollback, decision discarding, and plan resume functionality.
* HAL 9000 has contributed the PyYAML security upgrade (PR #11012 / issue #9055): added `pyyaml>=6.0.3` dependency constraint to address known YAML parsing vulnerabilities.
* HAL 9000 has contributed mandatory PR compliance checklist BDD tests for
implementation-pool-supervisor (#11052): created new feature file
`pr_compliance_checklist_pool_supervisor.feature` and step definitions file
covering all 8 mandatory checklist items (CHANGELOG.md, CONTRIBUTORS.md, commit
footer, CI passes, BDD tests, Epic reference, labels, milestone) to ensure
reproducible compliance verification for the implementation pool supervisor agent.
@@ -0,0 +1,58 @@
@mock_only
Feature: PR Compliance Checklist in Implementation Pool Supervisor
As an implementation pool supervisor
I want to pass a mandatory PR compliance checklist to every worker prompt
So that workers complete all required items before creating a PR and avoid systemic merge blockers
Background:
Given the implementation-pool-supervisor.md agent definition exists
Scenario: Pool supervisor contains PR Compliance Checklist section
When I read the implementation pool supervisor agent definition
Then the agent file includes the PR compliance checklist section
And the checklist is marked as MANDATORY
Scenario: Pool supervisor item 1 — CHANGELOG.md update required
When I read the implementation pool supervisor agent definition
Then the agent file includes a CHANGELOG.md checklist item
And the item instructs workers to add an entry under the Unreleased section
Scenario: Pool supervisor item 2 — CONTRIBUTORS.md update required
When I read the implementation pool supervisor agent definition
Then the agent file includes a CONTRIBUTORS.md checklist item
And the item instructs workers to add or update their contribution entry
Scenario: Pool supervisor item 3 — Commit footer required
When I read the implementation pool supervisor agent definition
Then the agent file includes a commit footer checklist item
And the item specifies the ISSUES CLOSED footer format
Scenario: Pool supervisor item 4 — CI must pass before PR creation
When I read the implementation pool supervisor agent definition
Then the agent file includes a CI passes checklist item
And the item instructs workers to verify all quality gates are green
Scenario: Pool supervisor item 5 — BDD/Behave tests required
When I read the implementation pool supervisor agent definition
Then the agent file includes a BDD tests checklist item
And the item instructs workers to add or update Behave feature files
Scenario: Pool supervisor item 6 — Epic reference required in PR description
When I read the implementation pool supervisor agent definition
Then the agent file includes an Epic reference / parent Epic checklist item
And the item instructs workers to reference the parent Epic issue number
Scenario: Pool supervisor item 7 — Labels must be applied
When I read the implementation pool supervisor agent definition
Then the agent file includes a labels checklist item
And the item instructs workers to apply labels via forgejo-label-manager
Scenario: Pool supervisor item 8 — Milestone must be assigned
When I read the implementation pool supervisor agent definition
Then the agent file includes a milestone checklist item
And the item instructs workers to assign the earliest open milestone
Scenario: All 8 checklist items are present in the pool supervisor
When I read the implementation pool supervisor agent definition
Then the agent file contains all 8 mandatory checklist items
@@ -0,0 +1,186 @@
"""Step definitions for PR compliance checklist in implementation pool supervisor."""
from pathlib import Path
from typing import Any
from behave import given, then, when
PROJECT_ROOT = Path(__file__).resolve().parents[2]
AGENT_DEF_PATH = PROJECT_ROOT / ".opencode" / "agents" / "implementation-pool-supervisor.md"
@given("the implementation-pool-supervisor.md agent definition exists")
def step_agent_def_exists(context: Any) -> None:
"""Verify the implementation pool supervisor agent definition file exists."""
assert AGENT_DEF_PATH.exists(), f"Agent definition not found at {AGENT_DEF_PATH}"
context.agent_def_path = AGENT_DEF_PATH
@when("I read the implementation pool supervisor agent definition")
def step_read_agent_def(context: Any) -> None:
"""Read the implementation pool supervisor agent definition."""
context.agent_def_content = AGENT_DEF_PATH.read_text(encoding="utf-8")
@then("the agent file includes the PR compliance checklist section")
def step_prompt_includes_checklist(context: Any) -> None:
"""Verify the agent file includes the PR compliance checklist."""
assert "PR Compliance Checklist" in context.agent_def_content, (
"Agent file does not include 'PR Compliance Checklist'"
)
@then("the checklist is marked as MANDATORY")
def step_checklist_is_mandatory(context: Any) -> None:
"""Verify the checklist is marked as MANDATORY."""
assert "MANDATORY" in context.agent_def_content, (
"PR Compliance Checklist is not marked as MANDATORY"
)
@then("the agent file includes a CHANGELOG.md checklist item")
def step_prompt_includes_changelog_item(context: Any) -> None:
"""Verify the agent file includes a CHANGELOG.md checklist item."""
assert "CHANGELOG.md" in context.agent_def_content, (
"Agent file does not include a CHANGELOG.md checklist item"
)
@then("the item instructs workers to add an entry under the Unreleased section")
def step_changelog_item_unreleased(context: Any) -> None:
"""Verify the CHANGELOG.md item mentions the Unreleased section."""
assert "[Unreleased]" in context.agent_def_content, (
"CHANGELOG.md checklist item does not mention the [Unreleased] section"
)
@then("the agent file includes a CONTRIBUTORS.md checklist item")
def step_prompt_includes_contributors_item(context: Any) -> None:
"""Verify the agent file includes a CONTRIBUTORS.md checklist item."""
assert "CONTRIBUTORS.md" in context.agent_def_content, (
"Agent file does not include a CONTRIBUTORS.md checklist item"
)
@then("the item instructs workers to add or update their contribution entry")
def step_contributors_item_add_update(context: Any) -> None:
"""Verify the CONTRIBUTORS.md item instructs workers to add or update."""
assert "add or update" in context.agent_def_content, (
"CONTRIBUTORS.md checklist item does not instruct workers to add or update"
)
@then("the agent file includes a commit footer checklist item")
def step_prompt_includes_commit_footer_item(context: Any) -> None:
"""Verify the agent file includes a commit footer checklist item."""
assert "Commit footer" in context.agent_def_content, (
"Agent file does not include a commit footer checklist item"
)
@then("the item specifies the ISSUES CLOSED footer format")
def step_commit_footer_issues_closed(context: Any) -> None:
"""Verify the commit footer item specifies the ISSUES CLOSED format."""
assert "ISSUES CLOSED" in context.agent_def_content, (
"Commit footer checklist item does not specify the ISSUES CLOSED format"
)
@then("the agent file includes a CI passes checklist item")
def step_prompt_includes_ci_item(context: Any) -> None:
"""Verify the agent file includes a CI passes checklist item."""
assert "CI passes" in context.agent_def_content, (
"Agent file does not include a CI passes checklist item"
)
@then("the item instructs workers to verify all quality gates are green")
def step_ci_item_quality_gates(context: Any) -> None:
"""Verify the CI item instructs workers to verify quality gates."""
assert "quality gates" in context.agent_def_content, (
"CI checklist item does not mention quality gates"
)
@then("the agent file includes a BDD tests checklist item")
def step_prompt_includes_bdd_item(context: Any) -> None:
"""Verify the agent file includes a BDD/Behave tests checklist item."""
assert "BDD/Behave tests" in context.agent_def_content, (
"Agent file does not include a BDD/Behave tests checklist item"
)
@then("the item instructs workers to add or update Behave feature files")
def step_bdd_item_feature_files(context: Any) -> None:
"""Verify the BDD item instructs workers to add or update feature files."""
assert "added or updated" in context.agent_def_content, (
"BDD checklist item does not instruct workers to add or update feature files"
)
@then("the agent file includes an Epic reference / parent Epic checklist item")
def step_prompt_includes_epic_item(context: Any) -> None:
"""Verify the agent file includes an Epic reference checklist item."""
assert "Epic reference" in context.agent_def_content or \
"parent Epic" in context.agent_def_content, (
"Agent file does not include an Epic reference checklist item"
)
@then("the item instructs workers to reference the parent Epic issue number")
def step_epic_item_parent_reference(context: Any) -> None:
"""Verify the Epic item instructs workers to reference the parent Epic."""
assert "parent Epic" in context.agent_def_content, (
"Epic checklist item does not instruct workers to reference the parent Epic"
)
@then("the agent file includes a labels checklist item")
def step_prompt_includes_labels_item(context: Any) -> None:
"""Verify the agent file includes a labels checklist item."""
assert "Labels" in context.agent_def_content, (
"Agent file does not include a labels checklist item"
)
@then("the item instructs workers to apply labels via forgejo-label-manager")
def step_labels_item_forgejo_label_manager(context: Any) -> None:
"""Verify the labels item instructs workers to use forgejo-label-manager."""
assert "forgejo-label-manager" in context.agent_def_content, (
"Labels checklist item does not mention forgejo-label-manager"
)
@then("the agent file includes a milestone checklist item")
def step_prompt_includes_milestone_item(context: Any) -> None:
"""Verify the agent file includes a milestone checklist item."""
assert "Milestone" in context.agent_def_content, (
"Agent file does not include a milestone checklist item"
)
@then("the item instructs workers to assign the earliest open milestone")
def step_milestone_item_earliest(context: Any) -> None:
"""Verify the milestone item instructs workers to assign the earliest open milestone."""
assert "earliest open milestone" in context.agent_def_content, (
"Milestone checklist item does not mention the earliest open milestone"
)
@then("the agent file contains all 8 mandatory checklist items")
def step_prompt_contains_all_8_items(context: Any) -> None:
"""Verify the agent file contains all 8 mandatory checklist items."""
required_items = [
"CHANGELOG.md",
"CONTRIBUTORS.md",
"ISSUES CLOSED",
"CI passes",
"BDD/Behave tests",
"Epic reference",
"forgejo-label-manager",
"earliest open milestone",
]
missing = [item for item in required_items if item not in context.agent_def_content]
assert not missing, (
f"Agent file is missing the following checklist items: {missing}"
)