feat(agents): add mandatory PR compliance checklist BDD tests for implementation-pool-supervisor #11181
+10
-1
@@ -30,7 +30,16 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
"ValueError") with no diagnostic detail, making production debugging
|
||||
impossible. The handler now includes the error message text and full
|
||||
traceback in the structlog warning entry. Removed `@tdd_expected_fail` tag
|
||||
from the TDD test so both scenarios run as normal regression guards. (#988)
|
||||
from the TDD test so both scenarios run as normal regression guards. (#988)
|
||||
|
||||
- **BDD tests added for implementation-pool-supervisor PR compliance checklist**
|
||||
(#11052): Extended BDD test coverage to verify all 8 mandatory PR compliance
|
||||
checklist items (CHANGELOG.md update, CONTRIBUTORS.md update, commit footer,
|
||||
CI verification, BDD/Behave tests, Epic reference, label application via
|
||||
forgejo-label-manager, and milestone assignment) are present in the
|
||||
`implementation-pool-supervisor.md` agent definition. New feature file
|
||||
`features/pr_compliance_checklist_pool_supervisor.feature` with dedicated step
|
||||
definitions ensures reproducible compliance verification.
|
||||
|
||||
### Added
|
||||
|
||||
|
||||
@@ -43,3 +43,10 @@ Below are some of the specific details of various contributions.
|
||||
* HAL 9000 has contributed database resource types (PostgreSQL, SQLite) with transaction-based sandbox strategy: implemented ``DatabaseResourceHandler`` providing full CRUD operations (`read`, `write`, `delete`, `list_children`) and connection validation with automatic credential masking for PostgreSQL and SQLite backends. Includes ``TransactionSandbox`` infrastructure wired into ``SandboxFactory``, BDD test coverage in ``features/database_resources.feature``, and Robot Framework integration tests in ``robot/database_resources.robot`` (PR #10591 / issue #8608, Epic #8568).
|
||||
* HAL 9000 has contributed the agents plan rollback command (PR #8674 / issue #8557): implemented checkpoint-based plan state restoration with the `agents plan rollback <plan-id> [<checkpoint-id>]` CLI command as part of Epic #8493, enabling plans to be restored to previous checkpoints, discarding post-checkpoint decisions, and resuming execution from the rolled-back state. Supported by `--yes/-y`, `--to-checkpoint`, and `--format/-f` flags. Includes comprehensive BDD test coverage (>= 97%) for rollback, decision discarding, and plan resume functionality.
|
||||
* HAL 9000 has contributed the PyYAML security upgrade (PR #11012 / issue #9055): added `pyyaml>=6.0.3` dependency constraint to address known YAML parsing vulnerabilities.
|
||||
|
||||
* HAL 9000 has contributed mandatory PR compliance checklist BDD tests for
|
||||
implementation-pool-supervisor (#11052): created new feature file
|
||||
`pr_compliance_checklist_pool_supervisor.feature` and step definitions file
|
||||
covering all 8 mandatory checklist items (CHANGELOG.md, CONTRIBUTORS.md, commit
|
||||
footer, CI passes, BDD tests, Epic reference, labels, milestone) to ensure
|
||||
reproducible compliance verification for the implementation pool supervisor agent.
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
@mock_only
|
||||
Feature: PR Compliance Checklist in Implementation Pool Supervisor
|
||||
|
||||
As an implementation pool supervisor
|
||||
I want to pass a mandatory PR compliance checklist to every worker prompt
|
||||
So that workers complete all required items before creating a PR and avoid systemic merge blockers
|
||||
|
||||
Background:
|
||||
Given the implementation-pool-supervisor.md agent definition exists
|
||||
|
||||
Scenario: Pool supervisor contains PR Compliance Checklist section
|
||||
When I read the implementation pool supervisor agent definition
|
||||
Then the agent file includes the PR compliance checklist section
|
||||
And the checklist is marked as MANDATORY
|
||||
|
||||
Scenario: Pool supervisor item 1 — CHANGELOG.md update required
|
||||
When I read the implementation pool supervisor agent definition
|
||||
Then the agent file includes a CHANGELOG.md checklist item
|
||||
And the item instructs workers to add an entry under the Unreleased section
|
||||
|
||||
Scenario: Pool supervisor item 2 — CONTRIBUTORS.md update required
|
||||
When I read the implementation pool supervisor agent definition
|
||||
Then the agent file includes a CONTRIBUTORS.md checklist item
|
||||
And the item instructs workers to add or update their contribution entry
|
||||
|
||||
Scenario: Pool supervisor item 3 — Commit footer required
|
||||
When I read the implementation pool supervisor agent definition
|
||||
Then the agent file includes a commit footer checklist item
|
||||
And the item specifies the ISSUES CLOSED footer format
|
||||
|
||||
Scenario: Pool supervisor item 4 — CI must pass before PR creation
|
||||
When I read the implementation pool supervisor agent definition
|
||||
Then the agent file includes a CI passes checklist item
|
||||
And the item instructs workers to verify all quality gates are green
|
||||
|
||||
Scenario: Pool supervisor item 5 — BDD/Behave tests required
|
||||
When I read the implementation pool supervisor agent definition
|
||||
Then the agent file includes a BDD tests checklist item
|
||||
And the item instructs workers to add or update Behave feature files
|
||||
|
||||
Scenario: Pool supervisor item 6 — Epic reference required in PR description
|
||||
When I read the implementation pool supervisor agent definition
|
||||
Then the agent file includes an Epic reference / parent Epic checklist item
|
||||
And the item instructs workers to reference the parent Epic issue number
|
||||
|
||||
Scenario: Pool supervisor item 7 — Labels must be applied
|
||||
When I read the implementation pool supervisor agent definition
|
||||
Then the agent file includes a labels checklist item
|
||||
And the item instructs workers to apply labels via forgejo-label-manager
|
||||
|
||||
Scenario: Pool supervisor item 8 — Milestone must be assigned
|
||||
When I read the implementation pool supervisor agent definition
|
||||
Then the agent file includes a milestone checklist item
|
||||
And the item instructs workers to assign the earliest open milestone
|
||||
|
||||
Scenario: All 8 checklist items are present in the pool supervisor
|
||||
When I read the implementation pool supervisor agent definition
|
||||
Then the agent file contains all 8 mandatory checklist items
|
||||
@@ -0,0 +1,186 @@
|
||||
"""Step definitions for PR compliance checklist in implementation pool supervisor."""
|
||||
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from behave import given, then, when
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parents[2]
|
||||
AGENT_DEF_PATH = PROJECT_ROOT / ".opencode" / "agents" / "implementation-pool-supervisor.md"
|
||||
|
||||
|
||||
@given("the implementation-pool-supervisor.md agent definition exists")
|
||||
def step_agent_def_exists(context: Any) -> None:
|
||||
"""Verify the implementation pool supervisor agent definition file exists."""
|
||||
assert AGENT_DEF_PATH.exists(), f"Agent definition not found at {AGENT_DEF_PATH}"
|
||||
context.agent_def_path = AGENT_DEF_PATH
|
||||
|
||||
|
||||
@when("I read the implementation pool supervisor agent definition")
|
||||
def step_read_agent_def(context: Any) -> None:
|
||||
"""Read the implementation pool supervisor agent definition."""
|
||||
context.agent_def_content = AGENT_DEF_PATH.read_text(encoding="utf-8")
|
||||
|
||||
|
||||
@then("the agent file includes the PR compliance checklist section")
|
||||
def step_prompt_includes_checklist(context: Any) -> None:
|
||||
"""Verify the agent file includes the PR compliance checklist."""
|
||||
assert "PR Compliance Checklist" in context.agent_def_content, (
|
||||
"Agent file does not include 'PR Compliance Checklist'"
|
||||
)
|
||||
|
||||
|
||||
@then("the checklist is marked as MANDATORY")
|
||||
def step_checklist_is_mandatory(context: Any) -> None:
|
||||
"""Verify the checklist is marked as MANDATORY."""
|
||||
assert "MANDATORY" in context.agent_def_content, (
|
||||
"PR Compliance Checklist is not marked as MANDATORY"
|
||||
)
|
||||
|
||||
|
||||
@then("the agent file includes a CHANGELOG.md checklist item")
|
||||
def step_prompt_includes_changelog_item(context: Any) -> None:
|
||||
"""Verify the agent file includes a CHANGELOG.md checklist item."""
|
||||
assert "CHANGELOG.md" in context.agent_def_content, (
|
||||
"Agent file does not include a CHANGELOG.md checklist item"
|
||||
)
|
||||
|
||||
|
||||
@then("the item instructs workers to add an entry under the Unreleased section")
|
||||
def step_changelog_item_unreleased(context: Any) -> None:
|
||||
"""Verify the CHANGELOG.md item mentions the Unreleased section."""
|
||||
assert "[Unreleased]" in context.agent_def_content, (
|
||||
"CHANGELOG.md checklist item does not mention the [Unreleased] section"
|
||||
)
|
||||
|
||||
|
||||
@then("the agent file includes a CONTRIBUTORS.md checklist item")
|
||||
def step_prompt_includes_contributors_item(context: Any) -> None:
|
||||
"""Verify the agent file includes a CONTRIBUTORS.md checklist item."""
|
||||
assert "CONTRIBUTORS.md" in context.agent_def_content, (
|
||||
"Agent file does not include a CONTRIBUTORS.md checklist item"
|
||||
)
|
||||
|
||||
|
||||
@then("the item instructs workers to add or update their contribution entry")
|
||||
def step_contributors_item_add_update(context: Any) -> None:
|
||||
"""Verify the CONTRIBUTORS.md item instructs workers to add or update."""
|
||||
assert "add or update" in context.agent_def_content, (
|
||||
"CONTRIBUTORS.md checklist item does not instruct workers to add or update"
|
||||
)
|
||||
|
||||
|
||||
@then("the agent file includes a commit footer checklist item")
|
||||
def step_prompt_includes_commit_footer_item(context: Any) -> None:
|
||||
"""Verify the agent file includes a commit footer checklist item."""
|
||||
assert "Commit footer" in context.agent_def_content, (
|
||||
"Agent file does not include a commit footer checklist item"
|
||||
)
|
||||
|
||||
|
||||
@then("the item specifies the ISSUES CLOSED footer format")
|
||||
def step_commit_footer_issues_closed(context: Any) -> None:
|
||||
"""Verify the commit footer item specifies the ISSUES CLOSED format."""
|
||||
assert "ISSUES CLOSED" in context.agent_def_content, (
|
||||
"Commit footer checklist item does not specify the ISSUES CLOSED format"
|
||||
)
|
||||
|
||||
|
||||
@then("the agent file includes a CI passes checklist item")
|
||||
def step_prompt_includes_ci_item(context: Any) -> None:
|
||||
"""Verify the agent file includes a CI passes checklist item."""
|
||||
assert "CI passes" in context.agent_def_content, (
|
||||
"Agent file does not include a CI passes checklist item"
|
||||
)
|
||||
|
||||
|
||||
@then("the item instructs workers to verify all quality gates are green")
|
||||
def step_ci_item_quality_gates(context: Any) -> None:
|
||||
"""Verify the CI item instructs workers to verify quality gates."""
|
||||
assert "quality gates" in context.agent_def_content, (
|
||||
"CI checklist item does not mention quality gates"
|
||||
)
|
||||
|
||||
|
||||
@then("the agent file includes a BDD tests checklist item")
|
||||
def step_prompt_includes_bdd_item(context: Any) -> None:
|
||||
"""Verify the agent file includes a BDD/Behave tests checklist item."""
|
||||
assert "BDD/Behave tests" in context.agent_def_content, (
|
||||
"Agent file does not include a BDD/Behave tests checklist item"
|
||||
)
|
||||
|
||||
|
||||
@then("the item instructs workers to add or update Behave feature files")
|
||||
def step_bdd_item_feature_files(context: Any) -> None:
|
||||
"""Verify the BDD item instructs workers to add or update feature files."""
|
||||
assert "added or updated" in context.agent_def_content, (
|
||||
"BDD checklist item does not instruct workers to add or update feature files"
|
||||
)
|
||||
|
||||
|
||||
@then("the agent file includes an Epic reference / parent Epic checklist item")
|
||||
def step_prompt_includes_epic_item(context: Any) -> None:
|
||||
"""Verify the agent file includes an Epic reference checklist item."""
|
||||
assert "Epic reference" in context.agent_def_content or \
|
||||
"parent Epic" in context.agent_def_content, (
|
||||
"Agent file does not include an Epic reference checklist item"
|
||||
)
|
||||
|
||||
|
||||
@then("the item instructs workers to reference the parent Epic issue number")
|
||||
def step_epic_item_parent_reference(context: Any) -> None:
|
||||
"""Verify the Epic item instructs workers to reference the parent Epic."""
|
||||
assert "parent Epic" in context.agent_def_content, (
|
||||
"Epic checklist item does not instruct workers to reference the parent Epic"
|
||||
)
|
||||
|
||||
|
||||
@then("the agent file includes a labels checklist item")
|
||||
def step_prompt_includes_labels_item(context: Any) -> None:
|
||||
"""Verify the agent file includes a labels checklist item."""
|
||||
assert "Labels" in context.agent_def_content, (
|
||||
"Agent file does not include a labels checklist item"
|
||||
)
|
||||
|
||||
|
||||
@then("the item instructs workers to apply labels via forgejo-label-manager")
|
||||
def step_labels_item_forgejo_label_manager(context: Any) -> None:
|
||||
"""Verify the labels item instructs workers to use forgejo-label-manager."""
|
||||
assert "forgejo-label-manager" in context.agent_def_content, (
|
||||
"Labels checklist item does not mention forgejo-label-manager"
|
||||
)
|
||||
|
||||
|
||||
@then("the agent file includes a milestone checklist item")
|
||||
def step_prompt_includes_milestone_item(context: Any) -> None:
|
||||
"""Verify the agent file includes a milestone checklist item."""
|
||||
assert "Milestone" in context.agent_def_content, (
|
||||
"Agent file does not include a milestone checklist item"
|
||||
)
|
||||
|
||||
|
||||
@then("the item instructs workers to assign the earliest open milestone")
|
||||
def step_milestone_item_earliest(context: Any) -> None:
|
||||
"""Verify the milestone item instructs workers to assign the earliest open milestone."""
|
||||
assert "earliest open milestone" in context.agent_def_content, (
|
||||
"Milestone checklist item does not mention the earliest open milestone"
|
||||
)
|
||||
|
||||
|
||||
@then("the agent file contains all 8 mandatory checklist items")
|
||||
def step_prompt_contains_all_8_items(context: Any) -> None:
|
||||
"""Verify the agent file contains all 8 mandatory checklist items."""
|
||||
required_items = [
|
||||
"CHANGELOG.md",
|
||||
"CONTRIBUTORS.md",
|
||||
"ISSUES CLOSED",
|
||||
"CI passes",
|
||||
"BDD/Behave tests",
|
||||
"Epic reference",
|
||||
"forgejo-label-manager",
|
||||
"earliest open milestone",
|
||||
]
|
||||
missing = [item for item in required_items if item not in context.agent_def_content]
|
||||
assert not missing, (
|
||||
f"Agent file is missing the following checklist items: {missing}"
|
||||
)
|
||||
Reference in New Issue
Block a user