cleveragents/cleveragents-core

Fork 3

BUG-HUNT: [documentation] SKILL.md documentation inconsistency with actual deploy script implementation #7302

New issue

Open

opened 2026-04-10 15:42:04 +00:00 by HAL9000 · 3 comments

HAL9000 commented

2026-04-10 15:42:04 +00:00

Owner

Background

The SKILL.md documentation for the deploy-to-staging agent skill describes a comprehensive deployment workflow with health checks and verification steps, but the actual implementation (deploy.py) is a placeholder stub that performs no real deployment actions. This creates a misleading user experience where agents and users following the documented workflow believe they are executing a real deployment when nothing substantive occurs.

Current Behavior

SKILL.md documents an 8-step deployment workflow including:

Step 5: "Run the deployment script from scripts/deploy.py"
Step 6: "Verify the deployment using the health check in references/runbook.md"

However, deploy.py contains only two print() statements and unconditionally returns 0:

def main() -> int:
    """Execute the deployment pipeline."""
    print("deploy-to-staging: starting deployment")
    print("deploy-to-staging: deployment complete")
    return 0

And references/runbook.md references health check endpoints (/health, /ready) that are never contacted by the script.

Specific inconsistencies:

SKILL.md step 5 references scripts/deploy.py as a real deployment step — the script is a no-op stub
SKILL.md step 6 references references/runbook.md health checks — the script never performs any health check
runbook.md documents /health and /ready endpoints — no integration exists in the script
No placeholder/template markers — nothing in the documentation warns users this is an example skeleton
No alignment between documented workflow and implementation reality — the gap is undocumented

Expected Behavior

Documentation should accurately describe what the implementation does, or the implementation should match the documented behavior. At minimum, the example should clearly indicate its placeholder/template nature so users and agents are not misled.

Impact

Users following SKILL.md instructions will receive misleading results — agents report false deployment success
Agents executing this skill will report success when no deployment occurred
Documentation does not match implementation reality, violating the project's Specification-First Development principle
Poor example for real deployment skill development — users may model production skills on this misleading template
Runbook health check references are unreachable dead references

Files Affected

examples/agent-skills/deploy-to-staging/SKILL.md
examples/agent-skills/deploy-to-staging/scripts/deploy.py
examples/agent-skills/deploy-to-staging/references/runbook.md

Suggested Fix

Update SKILL.md to clearly indicate this is a placeholder/template example
Add prominent warnings about example nature vs. production use
Align runbook.md health check references with actual script behavior, or add stub health check calls to the script
Add clear # EXAMPLE/TEMPLATE markers throughout the skill files
Consider implementing basic verification steps to match the documented workflow, or reduce the documented workflow to match the stub implementation

Acceptance Criteria

SKILL.md accurately describes what deploy.py actually does (or vice versa)
references/runbook.md health check references are consistent with script behavior
Clear placeholder/template markers are present if this remains an example skeleton
No misleading claims about deployment success when no deployment occurs
BDD scenarios tagged @tdd_issue, @tdd_expected_fail demonstrate the inconsistency before the fix
All nox stages pass with coverage ≥ 97%

Metadata

Branch: fix/documentation-skill-md-deploy-script-inconsistency
Commit Message: fix(examples): align SKILL.md documentation with deploy-to-staging script implementation
Milestone: (none — backlog)
Parent Epic: (to be linked — see orphan note below)

Subtasks

Audit all discrepancies between SKILL.md steps and deploy.py implementation
Audit references/runbook.md health check references against script behavior
Update SKILL.md to accurately reflect the placeholder/template nature of the skill
Add template/example markers to SKILL.md, deploy.py, and runbook.md
Align runbook.md health check documentation with actual script behavior
Write BDD scenario(s) demonstrating the documentation inconsistency (tagged @tdd_issue, @tdd_expected_fail)
Fix the inconsistency so all BDD scenarios pass
Confirm all nox stages pass with coverage ≥ 97%

Definition of Done

SKILL.md accurately describes what the implementation does
references/runbook.md health check references are consistent with script behavior
Clear placeholder/template markers present throughout the skill example files
No misleading success claims when no real deployment occurs
BDD tests tagged @tdd_issue prove the inconsistency and then pass after the fix
All nox stages pass
Coverage >= 97%

Backlog note: This issue was discovered during autonomous operation
on milestone v3.8.0. It does not block milestone completion and has been
placed in the backlog for human review and future milestone assignment.

Orphan note: No parent Epic was identified for this documentation inconsistency issue. A human reviewer should link this to the appropriate Epic covering agent-skill examples or documentation quality.

Automated by CleverAgents Bot
Supervisor: Bug Hunting | Agent: new-issue-creator

## Background The `SKILL.md` documentation for the `deploy-to-staging` agent skill describes a comprehensive deployment workflow with health checks and verification steps, but the actual implementation (`deploy.py`) is a placeholder stub that performs no real deployment actions. This creates a misleading user experience where agents and users following the documented workflow believe they are executing a real deployment when nothing substantive occurs. ## Current Behavior `SKILL.md` documents an 8-step deployment workflow including: - Step 5: "Run the deployment script from `scripts/deploy.py`" - Step 6: "Verify the deployment using the health check in `references/runbook.md`" However, `deploy.py` contains only two `print()` statements and unconditionally returns `0`: ```python def main() -> int: """Execute the deployment pipeline.""" print("deploy-to-staging: starting deployment") print("deploy-to-staging: deployment complete") return 0 ``` And `references/runbook.md` references health check endpoints (`/health`, `/ready`) that are never contacted by the script. Specific inconsistencies: 1. **SKILL.md step 5** references `scripts/deploy.py` as a real deployment step — the script is a no-op stub 2. **SKILL.md step 6** references `references/runbook.md` health checks — the script never performs any health check 3. **runbook.md** documents `/health` and `/ready` endpoints — no integration exists in the script 4. **No placeholder/template markers** — nothing in the documentation warns users this is an example skeleton 5. **No alignment between documented workflow and implementation reality** — the gap is undocumented ## Expected Behavior Documentation should accurately describe what the implementation does, or the implementation should match the documented behavior. At minimum, the example should clearly indicate its placeholder/template nature so users and agents are not misled. ## Impact - Users following `SKILL.md` instructions will receive misleading results — agents report false deployment success - Agents executing this skill will report success when no deployment occurred - Documentation does not match implementation reality, violating the project's Specification-First Development principle - Poor example for real deployment skill development — users may model production skills on this misleading template - Runbook health check references are unreachable dead references ## Files Affected - `examples/agent-skills/deploy-to-staging/SKILL.md` - `examples/agent-skills/deploy-to-staging/scripts/deploy.py` - `examples/agent-skills/deploy-to-staging/references/runbook.md` ## Suggested Fix 1. Update `SKILL.md` to clearly indicate this is a placeholder/template example 2. Add prominent warnings about example nature vs. production use 3. Align `runbook.md` health check references with actual script behavior, or add stub health check calls to the script 4. Add clear `# EXAMPLE/TEMPLATE` markers throughout the skill files 5. Consider implementing basic verification steps to match the documented workflow, or reduce the documented workflow to match the stub implementation ## Acceptance Criteria - `SKILL.md` accurately describes what `deploy.py` actually does (or vice versa) - `references/runbook.md` health check references are consistent with script behavior - Clear placeholder/template markers are present if this remains an example skeleton - No misleading claims about deployment success when no deployment occurs - BDD scenarios tagged `@tdd_issue`, `@tdd_expected_fail` demonstrate the inconsistency before the fix - All nox stages pass with coverage ≥ 97% ## Metadata - **Branch**: `fix/documentation-skill-md-deploy-script-inconsistency` - **Commit Message**: `fix(examples): align SKILL.md documentation with deploy-to-staging script implementation` - **Milestone**: *(none — backlog)* - **Parent Epic**: *(to be linked — see orphan note below)* ## Subtasks - [ ] Audit all discrepancies between `SKILL.md` steps and `deploy.py` implementation - [ ] Audit `references/runbook.md` health check references against script behavior - [ ] Update `SKILL.md` to accurately reflect the placeholder/template nature of the skill - [ ] Add template/example markers to `SKILL.md`, `deploy.py`, and `runbook.md` - [ ] Align `runbook.md` health check documentation with actual script behavior - [ ] Write BDD scenario(s) demonstrating the documentation inconsistency (tagged `@tdd_issue`, `@tdd_expected_fail`) - [ ] Fix the inconsistency so all BDD scenarios pass - [ ] Confirm all nox stages pass with coverage ≥ 97% ## Definition of Done - [ ] `SKILL.md` accurately describes what the implementation does - [ ] `references/runbook.md` health check references are consistent with script behavior - [ ] Clear placeholder/template markers present throughout the skill example files - [ ] No misleading success claims when no real deployment occurs - [ ] BDD tests tagged `@tdd_issue` prove the inconsistency and then pass after the fix - [ ] All nox stages pass - [ ] Coverage >= 97% > **Backlog note:** This issue was discovered during autonomous operation > on milestone v3.8.0. It does not block milestone completion and has been > placed in the backlog for human review and future milestone assignment. > **Orphan note:** No parent Epic was identified for this documentation inconsistency issue. A human reviewer should link this to the appropriate Epic covering agent-skill examples or documentation quality. --- **Automated by CleverAgents Bot** Supervisor: Bug Hunting | Agent: new-issue-creator

HAL9000 added the

labels

2026-04-10 15:48:02 +00:00

HAL9000 added

and removed

labels

2026-04-14 06:24:16 +00:00

HAL9000 commented

2026-04-14 06:24:16 +00:00

Author

Owner

✅ Verified — Documentation bug: SKILL.md inconsistency with deploy script. MoSCoW: Could-have. Priority: Low.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

✅ **Verified** — Documentation bug: SKILL.md inconsistency with deploy script. MoSCoW: Could-have. Priority: Low. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor

HAL9000 commented

2026-04-14 06:29:54 +00:00

Author

Owner

✅ Verified — Documentation bug: SKILL.md inconsistency with deploy script. MoSCoW: Could-have. Priority: Low.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor