forked from cleveragents/cleveragents-core
brent.edwards
093a74953f
Add comprehensive E2E test suite for M2 (Actor Graphs + Tool Sources) epic: - Behave BDD: 10 scenarios covering actor YAML loading, skill registry, tool lifecycle (discover/activate/execute/deactivate), and MCP stub - Robot Framework: 6 integration tests via CLI helper script - ASV benchmarks: 12 benchmarks for actor loading, skill registry, tool lifecycle, and MCP stub performance baselines - MCP stub server mock: in-process fake with 3 tools (search/fetch/transform) - Fixtures: hierarchical graph actor YAML + skill pack with tool refs and inline tools - Docs: updated testing.md with M2 smoke suite section Closes #169
87 lines
4.3 KiB
Gherkin
87 lines
4.3 KiB
Gherkin
Feature: M2 actor + tool source smoke suite
|
|
As a CleverAgents developer
|
|
I want to verify that actor loading, skill registry, tool lifecycle,
|
|
and MCP stub tool discovery work end-to-end
|
|
So that the M2 (Actor Graphs + Tool Sources) foundation is solid
|
|
|
|
# ── Actor YAML loading ───────────────────────────────────────────
|
|
|
|
Scenario: Load hierarchical actor from YAML fixture
|
|
Given I load the M2 hierarchical actor fixture
|
|
Then the M2 actor name should be "m2test/hierarchical-workflow"
|
|
And the M2 actor type should be "graph"
|
|
And the M2 actor should have 3 graph nodes
|
|
And the M2 actor entry node should be "planner"
|
|
And the M2 actor exit nodes should include "reviewer"
|
|
|
|
Scenario: Load hierarchical actor via ActorLoader from fixture directory
|
|
Given I create an M2 actor loader from the fixture directory
|
|
When I run M2 actor discovery
|
|
Then the M2 loader should find 1 actors
|
|
And the M2 loader should contain actor "m2test/hierarchical-workflow"
|
|
|
|
# ── Skill pack loading and registry ──────────────────────────────
|
|
|
|
Scenario: Load skill pack from YAML fixture
|
|
Given I load the M2 skill pack fixture
|
|
Then the M2 skill name should be "m2test/file-ops-pack"
|
|
And the M2 skill should have 2 tool references
|
|
And the M2 skill should have 1 inline tools
|
|
|
|
Scenario: Register skill pack and verify tool resolution
|
|
Given I register the M2 skill pack in a skill registry
|
|
When I resolve tools for M2 skill "m2test/file-ops-pack"
|
|
Then the M2 resolved tools should include a tool_ref entry
|
|
And the M2 resolved tools should include an inline entry
|
|
|
|
Scenario: Skill registry lists registered skills correctly
|
|
Given I register the M2 skill pack in a skill registry
|
|
When I list all M2 skills in the registry
|
|
Then the M2 skill list should contain "m2test/file-ops-pack"
|
|
And the M2 skill list should have 1 entries
|
|
|
|
Scenario: Invoke built-in tool through skill context
|
|
Given I create an M2 skill context with sandbox
|
|
When I register a tool invocation for "builtin/read_file" in the M2 context
|
|
Then the M2 skill context should have 1 invocation records
|
|
|
|
# ── Tool lifecycle ───────────────────────────────────────────────
|
|
|
|
Scenario: Tool lifecycle discover activate execute deactivate
|
|
Given I create an M2 tool runtime with a mock tool "m2test/echo"
|
|
And I create an M2 execution context with plan "m2-plan-001"
|
|
When I discover M2 tool "m2test/echo"
|
|
Then the M2 discovered descriptor name should be "m2test/echo"
|
|
When I activate M2 tool "m2test/echo"
|
|
And I execute M2 tool "m2test/echo" with params text "hello"
|
|
Then the M2 execution result should be successful
|
|
When I deactivate M2 tool "m2test/echo"
|
|
Then the M2 tool should be deactivated
|
|
|
|
Scenario: Tool registry registers and retrieves tools
|
|
Given I create an M2 tool registry
|
|
When I register an M2 tool spec "m2test/file-reader"
|
|
Then getting M2 tool "m2test/file-reader" should return a spec
|
|
And listing M2 tools should include "m2test/file-reader"
|
|
|
|
# ── MCP stub tool discovery and invocation ───────────────────────
|
|
|
|
Scenario: MCP stub tool discovery and invocation
|
|
Given I start the M2 MCP stub server
|
|
When I discover tools from the M2 MCP stub
|
|
Then the M2 stub should expose 3 tools
|
|
And the M2 stub tools should include "mcp/search"
|
|
And the M2 stub tools should include "mcp/fetch"
|
|
And the M2 stub tools should include "mcp/transform"
|
|
When I invoke M2 MCP stub tool "mcp/search" with query "test"
|
|
Then the M2 MCP stub result should contain results
|
|
And I stop the M2 MCP stub server
|
|
|
|
Scenario: MCP stub server lifecycle guards
|
|
Given I have an M2 MCP stub server that is stopped
|
|
When I try to discover tools from the stopped M2 stub
|
|
Then an M2 RuntimeError should be raised
|
|
When I start and invoke M2 MCP stub tool "mcp/fetch" with url "http://example.com"
|
|
Then the M2 MCP stub fetch result should have status 200
|
|
And the M2 MCP stub invocation log should have 1 entries
|