brent.edwards
093a74953f
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 16s
CI / lint (pull_request) Successful in 21s
CI / quality (pull_request) Successful in 27s
CI / security (pull_request) Successful in 56s
CI / typecheck (pull_request) Successful in 1m2s
CI / coverage (pull_request) Has been cancelled
CI / unit_tests (pull_request) Has been cancelled
CI / integration_tests (pull_request) Has been cancelled
CI / benchmark-regression (pull_request) Has been cancelled
CI / docker (pull_request) Has been cancelled
Add comprehensive E2E test suite for M2 (Actor Graphs + Tool Sources) epic: - Behave BDD: 10 scenarios covering actor YAML loading, skill registry, tool lifecycle (discover/activate/execute/deactivate), and MCP stub - Robot Framework: 6 integration tests via CLI helper script - ASV benchmarks: 12 benchmarks for actor loading, skill registry, tool lifecycle, and MCP stub performance baselines - MCP stub server mock: in-process fake with 3 tools (search/fetch/transform) - Fixtures: hierarchical graph actor YAML + skill pack with tool refs and inline tools - Docs: updated testing.md with M2 smoke suite section Closes #169
69 lines
3.2 KiB
Plaintext
69 lines
3.2 KiB
Plaintext
*** Settings ***
|
|
Documentation M2 actor + tool source CLI smoke suite
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${HELPER} ${CURDIR}/helper_m2_actor_tool_smoke.py
|
|
|
|
*** Test Cases ***
|
|
Actor Load Hierarchical YAML Fixture
|
|
[Documentation] Load hierarchical actor from M2 fixture YAML and verify
|
|
${result}= Run Process ${PYTHON} ${HELPER} actor-load-fixture cwd=${WORKSPACE} timeout=30s
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} actor-name: m2test/hierarchical-workflow
|
|
Should Contain ${result.stdout} actor-type: graph
|
|
Should Contain ${result.stdout} node-count: 3
|
|
|
|
Actor Discovery From Fixture Directory
|
|
[Documentation] Discover actors from the M2 fixtures directory
|
|
${result}= Run Process ${PYTHON} ${HELPER} actor-discover cwd=${WORKSPACE} timeout=30s
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} actor-count: 1
|
|
Should Contain ${result.stdout} actor-loaded: m2test/hierarchical-workflow
|
|
|
|
Skill Pack Load From YAML Fixture
|
|
[Documentation] Load skill pack from M2 fixture YAML and verify
|
|
${result}= Run Process ${PYTHON} ${HELPER} skill-load-fixture cwd=${WORKSPACE} timeout=30s
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} skill-name: m2test/file-ops-pack
|
|
Should Contain ${result.stdout} tool-ref-count: 2
|
|
Should Contain ${result.stdout} inline-tool-count: 1
|
|
|
|
Skill Registry And Tool Resolution
|
|
[Documentation] Register skill pack and verify tool resolution
|
|
${result}= Run Process ${PYTHON} ${HELPER} skill-registry cwd=${WORKSPACE} timeout=30s
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} skill-registered: m2test/file-ops-pack
|
|
Should Contain ${result.stdout} resolved-tools-ok
|
|
|
|
Tool Lifecycle Smoke
|
|
[Documentation] Verify discover/activate/execute/deactivate lifecycle
|
|
${result}= Run Process ${PYTHON} ${HELPER} tool-lifecycle cwd=${WORKSPACE} timeout=30s
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} discover-ok
|
|
Should Contain ${result.stdout} activate-ok
|
|
Should Contain ${result.stdout} execute-ok
|
|
Should Contain ${result.stdout} deactivate-ok
|
|
|
|
MCP Stub Discovery And Invocation
|
|
[Documentation] Start MCP stub, discover tools, invoke, and stop
|
|
${result}= Run Process ${PYTHON} ${HELPER} mcp-stub cwd=${WORKSPACE} timeout=30s
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} mcp-tool-count: 3
|
|
Should Contain ${result.stdout} mcp-invoke-ok: mcp/search
|
|
Should Contain ${result.stdout} mcp-stopped
|