aditya
cb82fc51df
Implemented AgentSkillSpec loader that parses SKILL.md frontmatter and progressive disclosure sections (discover/activate/deactivate) into structured SkillStep objects with stable 1-based ordering. Mapped Agent Skills to AgentSkillToolDescriptor with namespaced naming (namespace/short_name), source="agent_skill", read-only defaults, and AgentSkillResourceSlot bindings for scripts/, references/, and assets/ directories. All resource slots are unconditionally read_only. Added explicit validation for missing frontmatter fields (name, description) and invalid namespace format with actionable error messages. Added docs/reference/agent_skills.md covering folder layout, SKILL.md parsing rules, progressive disclosure model, and tool mapping. Added Behave scenarios covering valid/invalid SKILL.md parsing, namespaced naming, step ordering, missing frontmatter errors, progressive disclosure lifecycle, tool mapping, and resource binding slots. Added Robot Framework integration tests using the deploy-to-staging example skill folder (robot/agent_skills_loader.robot). Added ASV benchmarks for parsing throughput, folder load, progressive disclosure lifecycle, and resource listing (benchmarks/agent_skills_loader_bench.py). ISSUES CLOSED: #160
89 lines
4.4 KiB
Plaintext
89 lines
4.4 KiB
Plaintext
*** Settings ***
|
|
Documentation Integration smoke tests for Agent Skills Standard loader
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${HELPER} ${CURDIR}/helper_agent_skills_loader.py
|
|
${SAMPLE_SKILL} ${CURDIR}/../examples/agent-skills/deploy-to-staging
|
|
|
|
*** Test Cases ***
|
|
Load Sample Agent Skills Folder
|
|
[Documentation] Load the example deploy-to-staging skill folder and verify name/description
|
|
${result}= Run Process ${PYTHON} ${HELPER} load-skill ${SAMPLE_SKILL} cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} agent-skills-load-ok
|
|
|
|
Discover Returns Metadata Only
|
|
[Documentation] discover() should return name and description but no body
|
|
${result}= Run Process ${PYTHON} ${HELPER} discover ${SAMPLE_SKILL} cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} agent-skills-discover-ok
|
|
|
|
Activate Loads Full Body
|
|
[Documentation] activate() should load the full SKILL.md body
|
|
${result}= Run Process ${PYTHON} ${HELPER} activate ${SAMPLE_SKILL} cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} agent-skills-activate-ok
|
|
|
|
Tool Descriptor Has Correct Fields
|
|
[Documentation] to_tool_descriptor() should produce a descriptor with source=agent_skill and read_only=True
|
|
${result}= Run Process ${PYTHON} ${HELPER} tool-descriptor ${SAMPLE_SKILL} cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} agent-skills-tool-descriptor-ok
|
|
|
|
List Resources Finds Scripts And References
|
|
[Documentation] list_resources() should find files in scripts/ and references/
|
|
${result}= Run Process ${PYTHON} ${HELPER} list-resources ${SAMPLE_SKILL} cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} agent-skills-list-resources-ok
|
|
Should Contain ${result.stdout} deploy.py
|
|
Should Contain ${result.stdout} runbook.md
|
|
|
|
Invalid Folder Without SKILL.md Fails Gracefully
|
|
[Documentation] Loading a folder with no SKILL.md should fail with an informative error
|
|
${no_skill_dir}= Join Path ${TEMPDIR} no_skill_md_robot_test
|
|
Create Directory ${no_skill_dir}
|
|
${result}= Run Process ${PYTHON} ${HELPER} invalid-skill ${no_skill_dir} cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} agent-skills-invalid-ok
|
|
|
|
Non-Existent Folder Fails Gracefully
|
|
[Documentation] Loading a non-existent path should fail with an informative error
|
|
${result}= Run Process ${PYTHON} ${HELPER} invalid-skill /tmp/__nonexistent_skill_xyz__ cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} agent-skills-invalid-ok
|
|
|
|
Structured Steps Are Parsed From Frontmatter
|
|
[Documentation] steps: list in SKILL.md frontmatter → SkillStep objects with stable index ordering
|
|
${result}= Run Process ${PYTHON} ${HELPER} steps ${SAMPLE_SKILL} cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} agent-skills-steps-ok
|
|
Should Contain ${result.stdout} step[1]
|
|
|
|
Resource Binding Slots Are Read-Only
|
|
[Documentation] Resource directories (scripts/, references/) produce read-only binding slots
|
|
${result}= Run Process ${PYTHON} ${HELPER} resource-slots ${SAMPLE_SKILL} cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} agent-skills-resource-slots-ok
|
|
Should Contain ${result.stdout} access=read_only
|