UAT: `SkillAgentFolderSchema` missing `sandbox_policy` field — spec-required Agent Skills sandbox configuration is silently ignored #2116

New issue

Open

opened 2026-04-03 04:12:40 +00:00 by freemo · 1 comment

freemo commented

2026-04-03 04:12:40 +00:00

Owner

Metadata

Branch: fix/skills-skill-agent-folder-schema-sandbox-policy
Commit Message: fix(skills): add sandbox_policy field to SkillAgentFolderSchema
Milestone: v3.7.0
Parent Epic: #392

Background

The SkillAgentFolderSchema in src/cleveragents/skills/schema.py only defines path and name fields, but the specification requires a sandbox_policy field for each Agent Skills Standard folder reference. Because extra="forbid" is set on the model, any spec-compliant YAML that includes sandbox_policy is rejected with a Pydantic ValidationError, meaning users cannot configure sandbox policies for Agent Skills as the spec requires.

Steps to Reproduce

Create a skill YAML file using the spec-compliant sandbox_policy field:

name: local/test-skill
description: Test skill
agent_skill_folders:
  - path: ./skills/my-skill
    sandbox_policy: none

Call SkillConfigSchema.from_yaml(yaml_content)
Observe: Pydantic raises ValidationError: Extra inputs are not permitted for sandbox_policy

Expected Behaviour (from spec)

The skill YAML configuration should accept sandbox_policy in each agent_skills entry:

skill:
  name: local/devops-toolkit
  agent_skills:
    - path: ./skills/code-review-checklist
      sandbox_policy: none

The spec defines sandbox_policy as controlling how scripts bundled with the Agent Skill are executed. Valid values: none, restricted, full.

Actual Behaviour

class SkillAgentFolderSchema(BaseModel):
    path: str = Field(...)
    name: str | None = Field(default=None, ...)
    model_config = ConfigDict(extra="forbid")

No sandbox_policy field is present. Any YAML including sandbox_policy raises a Pydantic validation error.

Code Location

src/cleveragents/skills/schema.py — SkillAgentFolderSchema class

Severity

High — Users cannot configure sandbox policies for Agent Skills as required by the spec. Any spec-compliant YAML with sandbox_policy is rejected outright.

Subtasks

Review docs/specification.md skill YAML configuration section to confirm all required sandbox_policy values (none, restricted, full) and any default value
Add sandbox_policy field to SkillAgentFolderSchema in src/cleveragents/skills/schema.py with correct type annotation and default value
Ensure static typing passes (nox -e typecheck)
Write Behave unit tests in features/ covering all valid sandbox_policy values and the rejection of invalid values
Write Behave unit test confirming that omitting sandbox_policy uses the correct default (backward-compatible)
Update any related integration tests in robot/ if applicable
Verify nox -e lint passes
Verify nox -e unit_tests passes
Verify nox -e coverage_report shows coverage >= 97%

Definition of Done

sandbox_policy field added to SkillAgentFolderSchema with correct type, valid values, and default
Spec-compliant YAML with sandbox_policy is accepted without error
YAML omitting sandbox_policy continues to work (backward-compatible)
YAML with an invalid sandbox_policy value raises a clear validation error
All Behave unit tests pass (nox -e unit_tests)
All integration tests pass (nox -e integration_tests)
All nox stages pass
Coverage >= 97%
Commit fix(skills): add sandbox_policy field to SkillAgentFolderSchema pushed to branch fix/skills-skill-agent-folder-schema-sandbox-policy
PR merged and this issue closed

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-uat-tester

## Metadata - **Branch**: `fix/skills-skill-agent-folder-schema-sandbox-policy` - **Commit Message**: `fix(skills): add sandbox_policy field to SkillAgentFolderSchema` - **Milestone**: v3.7.0 - **Parent Epic**: #392 ## Background The `SkillAgentFolderSchema` in `src/cleveragents/skills/schema.py` only defines `path` and `name` fields, but the specification requires a `sandbox_policy` field for each Agent Skills Standard folder reference. Because `extra="forbid"` is set on the model, any spec-compliant YAML that includes `sandbox_policy` is rejected with a Pydantic `ValidationError`, meaning users cannot configure sandbox policies for Agent Skills as the spec requires. ## Steps to Reproduce 1. Create a skill YAML file using the spec-compliant `sandbox_policy` field: ```yaml name: local/test-skill description: Test skill agent_skill_folders: - path: ./skills/my-skill sandbox_policy: none ``` 2. Call `SkillConfigSchema.from_yaml(yaml_content)` 3. Observe: Pydantic raises `ValidationError: Extra inputs are not permitted` for `sandbox_policy` ## Expected Behaviour (from spec) The skill YAML configuration should accept `sandbox_policy` in each `agent_skills` entry: ```yaml skill: name: local/devops-toolkit agent_skills: - path: ./skills/code-review-checklist sandbox_policy: none ``` The spec defines `sandbox_policy` as controlling how scripts bundled with the Agent Skill are executed. Valid values: `none`, `restricted`, `full`. ## Actual Behaviour ```python class SkillAgentFolderSchema(BaseModel): path: str = Field(...) name: str | None = Field(default=None, ...) model_config = ConfigDict(extra="forbid") ``` No `sandbox_policy` field is present. Any YAML including `sandbox_policy` raises a Pydantic validation error. ## Code Location - `src/cleveragents/skills/schema.py` — `SkillAgentFolderSchema` class ## Severity **High** — Users cannot configure sandbox policies for Agent Skills as required by the spec. Any spec-compliant YAML with `sandbox_policy` is rejected outright. ## Subtasks - [ ] Review `docs/specification.md` skill YAML configuration section to confirm all required `sandbox_policy` values (`none`, `restricted`, `full`) and any default value - [ ] Add `sandbox_policy` field to `SkillAgentFolderSchema` in `src/cleveragents/skills/schema.py` with correct type annotation and default value - [ ] Ensure static typing passes (`nox -e typecheck`) - [ ] Write Behave unit tests in `features/` covering all valid `sandbox_policy` values and the rejection of invalid values - [ ] Write Behave unit test confirming that omitting `sandbox_policy` uses the correct default (backward-compatible) - [ ] Update any related integration tests in `robot/` if applicable - [ ] Verify `nox -e lint` passes - [ ] Verify `nox -e unit_tests` passes - [ ] Verify `nox -e coverage_report` shows coverage >= 97% ## Definition of Done - [ ] `sandbox_policy` field added to `SkillAgentFolderSchema` with correct type, valid values, and default - [ ] Spec-compliant YAML with `sandbox_policy` is accepted without error - [ ] YAML omitting `sandbox_policy` continues to work (backward-compatible) - [ ] YAML with an invalid `sandbox_policy` value raises a clear validation error - [ ] All Behave unit tests pass (`nox -e unit_tests`) - [ ] All integration tests pass (`nox -e integration_tests`) - [ ] All nox stages pass - [ ] Coverage >= 97% - [ ] Commit `fix(skills): add sandbox_policy field to SkillAgentFolderSchema` pushed to branch `fix/skills-skill-agent-folder-schema-sandbox-policy` - [ ] PR merged and this issue closed --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-uat-tester

freemo added this to the v3.7.0 milestone

2026-04-03 04:12:45 +00:00

freemo added a new dependency

2026-04-03 04:13:13 +00:00

#392 Epic: Actor YAML & Compiler

freemo referenced this issue

2026-04-03 04:20:11 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.7.0 — Session Tracker #2023

freemo referenced this issue

2026-04-03 04:22:00 +00:00

[CA-AUTO] UAT Pool Supervisor — v3.7.0 — Session Tracker #2023

freemo self-assigned this

2026-04-03 16:58:05 +00:00

freemo commented

2026-04-03 18:28:15 +00:00

Author

Owner

MoSCoW classification: Should Have

Rationale: This issue addresses a spec requirement or important quality improvement. It should be included in the milestone if possible.

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: ca-project-owner

MoSCoW classification: **Should Have** Rationale: This issue addresses a spec requirement or important quality improvement. It should be included in the milestone if possible. --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: ca-project-owner