HAL9000
f808abff86
Fix JSON syntax errors in .devcontainer/devcontainer.json (removed
invalid JS-style // comments) and .devcontainer/opencode.json (removed
90+ trailing commas). Apply auto-fixes for end-of-file and trailing
whitespace issues across 100+ files. Fix SIM105 ruff violations in
benchmarks/core_circuit_breaker_bench.py (use contextlib.suppress).
Note: The security fix from issue #7478 (validate_path startswith bypass)
was already delivered to master in commit e18ac5f2. This PR as currently
structured is non-atomic (35 commits across 10+ issues) and needs
significant restructure before merge. This commit only addresses the
CI/pre-commit failures.
ISSUES CLOSED: #7478
1362 lines
58 KiB
Gherkin
1362 lines
58 KiB
Gherkin
Feature: Consolidated Actor
|
|
Combined scenarios from: actor_compiler, actor_compiler_coverage, actor_config_new_coverage, actor_hierarchy, actor_registry_coverage, actor_registry_full_coverage, actor_registry_persistence, actor_runtime, m2_actor_tool_smoke
|
|
|
|
# ============================================================
|
|
# Originally from: actor_compiler.feature
|
|
# Feature: Actor compiler translates hierarchical actor configs to LangGraph
|
|
# ============================================================
|
|
|
|
Scenario: Compile a simple two-node graph actor
|
|
Given a GRAPH actor config with a linear two-node topology
|
|
When I compile the actor config
|
|
Then the compilation should succeed
|
|
And the compiled actor should have 2 nodes
|
|
And the compiled actor entry point should be "planner"
|
|
And the compilation metadata should list node IDs "executor" and "planner"
|
|
|
|
|
|
Scenario: Compile a graph with a tool node
|
|
Given a GRAPH actor config with an agent and a tool node
|
|
When I compile the actor config
|
|
Then the compilation should succeed
|
|
And the compilation metadata should have 1 tool node
|
|
|
|
|
|
Scenario: Compile a graph with LSP bindings on a node
|
|
Given a GRAPH actor config with LSP bindings on the agent node
|
|
When I compile the actor config
|
|
Then the compilation should succeed
|
|
And the compilation metadata should have 1 LSP binding
|
|
And the LSP binding should reference server "local/pyright"
|
|
|
|
|
|
Scenario: Compile a graph with a conditional node
|
|
Given a GRAPH actor config with a conditional routing node
|
|
When I compile the actor config
|
|
Then the compilation should succeed
|
|
And the compiled actor should have 3 nodes
|
|
|
|
|
|
Scenario: Compile a graph with a subgraph reference
|
|
Given a GRAPH actor config with a subgraph node referencing "workflows/inner"
|
|
When I compile the actor config with a resolver
|
|
Then the compilation should succeed
|
|
And the compilation metadata subgraph refs should map "sub" to "workflows/inner"
|
|
|
|
|
|
Scenario: Compile a single-node graph actor
|
|
Given a GRAPH actor config with a single node
|
|
When I compile the actor config
|
|
Then the compilation should succeed
|
|
And the compiled actor should have 1 nodes
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Subgraph resolution and cycle detection
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
|
|
Scenario: Detect cross-actor subgraph cycle
|
|
Given a GRAPH actor config "workflows/outer" referencing "workflows/inner"
|
|
And a resolver where "workflows/inner" references "workflows/outer"
|
|
When I compile the actor config with a resolver
|
|
Then the compilation should fail with SubgraphCycleError
|
|
And the compiler error message should contain "cycle"
|
|
|
|
|
|
Scenario: Accept acyclic subgraph chain
|
|
Given a GRAPH actor config "workflows/top" referencing "workflows/mid"
|
|
And a resolver where "workflows/mid" has no subgraph nodes
|
|
When I compile the actor config with a resolver
|
|
Then the compilation should succeed
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Missing node and invalid entry/exit errors
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
|
|
Scenario: Reject non-GRAPH actor type
|
|
Given an LLM actor config
|
|
When I attempt to compile the non-graph actor
|
|
Then the compilation should fail with ActorCompilationError
|
|
And the compiler error message should contain "GRAPH"
|
|
|
|
|
|
Scenario: Reject GRAPH actor without route
|
|
Given a GRAPH actor config with route set to None
|
|
When I attempt to compile the routeless actor
|
|
Then the compilation should fail with ActorCompilationError
|
|
And the compiler error message should contain "no route"
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Metadata inspection
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
|
|
Scenario: Metadata lists all node IDs sorted
|
|
Given a GRAPH actor config with nodes "alpha", "beta", and "gamma"
|
|
When I compile the actor config
|
|
Then the compilation should succeed
|
|
And the metadata node IDs should be sorted alphabetically
|
|
|
|
|
|
Scenario: Metadata records entry and exit nodes
|
|
Given a GRAPH actor config with entry "start_node" and exit "end_node"
|
|
When I compile the actor config
|
|
Then the compilation should succeed
|
|
And the metadata entry node should be "start_node"
|
|
And the metadata exit nodes should contain "end_node"
|
|
|
|
|
|
Scenario: Compilation metadata is JSON-serializable
|
|
Given a GRAPH actor config with LSP bindings on the agent node
|
|
When I compile the actor config
|
|
Then the compilation metadata should be JSON-serializable
|
|
|
|
|
|
# ============================================================
|
|
# Originally from: actor_compiler_coverage.feature
|
|
# Feature: Actor compiler full coverage for node mapping, edge mapping, LSP bindings, and error paths
|
|
# ============================================================
|
|
|
|
Scenario: Compile a graph with all four node types mapped correctly
|
|
Given a GRAPH actor config containing AGENT, TOOL, CONDITIONAL, and SUBGRAPH nodes
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should succeed without errors
|
|
And the compiled node "agent_node" should have LangGraph type "agent"
|
|
And the compiled node "tool_node" should have LangGraph type "tool"
|
|
And the compiled node "cond_node" should have LangGraph type "conditional"
|
|
And the compiled node "sub_node" should have LangGraph type "subgraph"
|
|
|
|
|
|
Scenario: Agent node config maps agent field to NodeConfig
|
|
Given a GRAPH actor config with an agent node configured as "my_agent"
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should succeed without errors
|
|
And the compiled node "agent_node" should have agent set to "my_agent"
|
|
|
|
|
|
Scenario: Conditional node config maps function field to NodeConfig
|
|
Given a GRAPH actor config with a conditional node configured with function "route_fn"
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should succeed without errors
|
|
And the compiled node "cond_node" should have function set to "route_fn"
|
|
|
|
|
|
Scenario: Tool node config maps tools list to NodeConfig
|
|
Given a GRAPH actor config with a tool node configured with tools "lint,format"
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should succeed without errors
|
|
And the compiled node "tool_node" should have tools "lint" and "format"
|
|
|
|
|
|
Scenario: Subgraph node config maps actor_ref to NodeConfig subgraph field
|
|
Given a GRAPH actor config with a subgraph node referencing actor "workflows/child"
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should succeed without errors
|
|
And the compiled node "sub_node" should have subgraph set to "workflows/child"
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Edge mapping (_map_edge)
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
|
|
Scenario: Edge without condition compiles to null condition field
|
|
Given a GRAPH actor config with an unconditional edge from "a" to "b"
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should succeed without errors
|
|
And the compiled edge from "a" to "b" should have a null condition
|
|
|
|
|
|
Scenario: Edge with condition compiles to expression dict
|
|
Given a GRAPH actor config with a conditional edge from "a" to "b" with condition "state.get('done')"
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should succeed without errors
|
|
And the compiled edge from "a" to "b" should have condition expression "state.get('done')"
|
|
|
|
|
|
Scenario: Edge priority is recorded in compiled edge metadata
|
|
Given a GRAPH actor config with an edge from "a" to "b" with priority 5
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should succeed without errors
|
|
And the compiled edge from "a" to "b" should have metadata priority 5
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# LSP binding extraction edge cases (_extract_lsp_bindings)
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
|
|
Scenario: Non-list LSP bindings config value is safely ignored
|
|
Given a GRAPH actor config where lsp_bindings is a string instead of a list
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should succeed without errors
|
|
And the compilation metadata should have 0 LSP bindings
|
|
|
|
|
|
Scenario: Non-dict entry in LSP bindings list is skipped
|
|
Given a GRAPH actor config where lsp_bindings contains a non-dict entry
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should succeed without errors
|
|
And the compilation metadata should have 0 LSP bindings
|
|
|
|
|
|
Scenario: LSP binding with empty server name is skipped
|
|
Given a GRAPH actor config where an LSP binding has an empty server name
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should succeed without errors
|
|
And the compilation metadata should have 0 LSP bindings
|
|
|
|
|
|
Scenario: LSP binding with non-string server name is skipped
|
|
Given a GRAPH actor config where an LSP binding has a numeric server name
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should succeed without errors
|
|
And the compilation metadata should have 0 LSP bindings
|
|
|
|
|
|
Scenario: LSP binding with missing server name key is skipped
|
|
Given a GRAPH actor config where an LSP binding is missing the server name key
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should succeed without errors
|
|
And the compilation metadata should have 0 LSP bindings
|
|
|
|
|
|
Scenario: Multiple valid LSP bindings are extracted from a single node
|
|
Given a GRAPH actor config with two valid LSP bindings on one node
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should succeed without errors
|
|
And the compilation metadata should have 2 LSP bindings
|
|
And the LSP binding servers should include "local/pyright" and "local/ts"
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Subgraph cycle detection edge cases (_detect_subgraph_cycles)
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
|
|
Scenario: Subgraph node with empty actor_ref is skipped during cycle detection
|
|
Given a GRAPH actor config with a subgraph node that has an empty actor_ref
|
|
When I compile the full-coverage actor config with a resolver
|
|
Then the compilation should succeed without errors
|
|
|
|
|
|
Scenario: Resolver returning None for a referenced actor skips that node
|
|
Given a GRAPH actor config with a subgraph referencing actor name "workflows/missing"
|
|
And a cycle-detection resolver that returns None for all names
|
|
When I compile the full-coverage actor config with a resolver
|
|
Then the compilation should succeed without errors
|
|
|
|
|
|
Scenario: Referenced actor that is not GRAPH type is skipped during cycle detection
|
|
Given a GRAPH actor config with a subgraph referencing actor name "tools/helper"
|
|
And a cycle-detection resolver where "tools/helper" is an LLM type actor
|
|
When I compile the full-coverage actor config with a resolver
|
|
Then the compilation should succeed without errors
|
|
|
|
|
|
Scenario: Referenced actor with no route definition is skipped during cycle detection
|
|
Given a GRAPH actor config with a subgraph referencing actor name "workflows/no-route"
|
|
And a cycle-detection resolver where "workflows/no-route" is a GRAPH actor with no route
|
|
When I compile the full-coverage actor config with a resolver
|
|
Then the compilation should succeed without errors
|
|
|
|
|
|
Scenario: Deep three-level subgraph cycle is detected
|
|
Given a GRAPH actor config named "workflows/alpha" with subgraph ref to "workflows/beta"
|
|
And a cycle-detection resolver where "workflows/beta" references "workflows/gamma" and "workflows/gamma" references "workflows/alpha"
|
|
When I compile the full-coverage actor config with a resolver
|
|
Then the compilation should fail with a SubgraphCycleError
|
|
And the compilation error message should mention "cycle"
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# compile_actor error paths
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
|
|
Scenario: Intra-graph cycle raises SubgraphCycleError during compilation
|
|
Given a GRAPH actor config with a cyclic route that bypasses schema validation
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should fail with a SubgraphCycleError
|
|
And the compilation error message should mention "intra-graph"
|
|
|
|
|
|
Scenario: Edge referencing missing from_node raises MissingNodeError
|
|
Given a GRAPH actor config with an edge whose from_node does not exist in the node list
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should fail with a MissingNodeError
|
|
And the compilation error message should mention "from_node"
|
|
|
|
|
|
Scenario: Edge referencing missing to_node raises MissingNodeError
|
|
Given a GRAPH actor config with an edge whose to_node does not exist in the node list
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should fail with a MissingNodeError
|
|
And the compilation error message should mention "to_node"
|
|
|
|
|
|
Scenario: Invalid entry node raises InvalidEntryExitError
|
|
Given a GRAPH actor config whose entry node does not match any compiled node
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should fail with an InvalidEntryExitError
|
|
And the compilation error message should mention "Entry node"
|
|
|
|
|
|
Scenario: Invalid exit node raises InvalidEntryExitError
|
|
Given a GRAPH actor config whose exit node does not match any compiled node
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should fail with an InvalidEntryExitError
|
|
And the compilation error message should mention "Exit node"
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Model defaults and result bundle
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
|
|
Scenario: CompilationMetadata defaults to empty collections
|
|
Given a freshly constructed CompilationMetadata with no arguments
|
|
Then the metadata node_ids should be an empty list
|
|
And the metadata tool_nodes should be an empty list
|
|
And the metadata lsp_bindings should be an empty list
|
|
And the metadata subgraph_refs should be an empty dict
|
|
And the metadata entry_node should be an empty string
|
|
And the metadata exit_nodes should be an empty list
|
|
|
|
|
|
Scenario: CompiledActor preserves actor name and entry point from config
|
|
Given a GRAPH actor config named "workflows/test" with entry "start"
|
|
When I compile the full-coverage actor config
|
|
Then the compilation should succeed without errors
|
|
And the compiled actor name should be "workflows/test"
|
|
And the compiled actor entry_point field should be "start"
|
|
|
|
|
|
# ============================================================
|
|
# Originally from: actor_config_new_coverage.feature
|
|
# Feature: Actor configuration new coverage
|
|
# ============================================================
|
|
|
|
Scenario: load_blob_from_file loads JSON config
|
|
Given a temporary JSON config file with provider and model
|
|
When I load the blob from the JSON file
|
|
Then the blob should contain provider and model keys
|
|
|
|
|
|
Scenario: load_blob_from_file raises for missing file
|
|
When I load the blob from a nonexistent path
|
|
Then an actor config ValueError should mention "not found"
|
|
|
|
|
|
Scenario: load_blob_from_file raises for non-dict JSON
|
|
Given a temporary JSON config file with a list
|
|
When I load the blob from the list JSON file
|
|
Then an actor config ValueError should mention "object"
|
|
|
|
|
|
Scenario: load_blob_from_file returns empty dict for null JSON
|
|
Given a temporary JSON config file with null content
|
|
When I load the blob from the null JSON file
|
|
Then the blob should be an empty dict
|
|
|
|
|
|
Scenario: load_blob_from_file falls back to YAML for invalid JSON
|
|
Given a temporary YAML-only config file with provider and model
|
|
When I load the blob from the YAML file
|
|
Then the blob should contain provider and model keys
|
|
|
|
|
|
Scenario: _load_v2_yaml_content parses plain YAML
|
|
When I load v2 YAML content without templates
|
|
Then the v2 result should be a dict with expected keys
|
|
|
|
|
|
Scenario: _load_v2_yaml_content handles empty YAML
|
|
When I load v2 YAML content that is empty
|
|
Then the v2 result should be an empty dict
|
|
|
|
|
|
Scenario: _load_v2_yaml_content rejects non-dict YAML
|
|
When I load v2 YAML content that is a list
|
|
Then an actor config ValueError should mention "object"
|
|
|
|
|
|
Scenario: _load_v2_yaml_content processes templated YAML
|
|
When I load v2 YAML content with Jinja2 templates
|
|
Then the actor config result should be a dict
|
|
|
|
|
|
Scenario: _restore_template_syntax restores markers in system_prompt
|
|
When I restore template syntax in a config with protected markers
|
|
Then the system_prompt should have Jinja2 syntax restored
|
|
|
|
|
|
Scenario: _restore_template_syntax recurses into lists
|
|
When I restore template syntax in a list config
|
|
Then nested list items should have markers restored
|
|
|
|
|
|
Scenario: _interpolate_env_vars substitutes environment variables
|
|
Given environment variable "TEST_ACTOR_VAR" is set to "openai"
|
|
When I interpolate env vars in config with "${TEST_ACTOR_VAR}"
|
|
Then the interpolated value should be "openai"
|
|
|
|
|
|
Scenario: _interpolate_env_vars uses default when env var missing
|
|
When I interpolate env vars with "${MISSING_VAR:fallback}"
|
|
Then the interpolated value should be "fallback"
|
|
|
|
|
|
Scenario: _interpolate_env_vars raises when env var missing without default
|
|
When I interpolate env vars with "${TOTALLY_MISSING_VAR}" and no default
|
|
Then an actor config ValueError should mention "not set"
|
|
|
|
|
|
Scenario: _interpolate_env_vars converts boolean strings
|
|
When I interpolate env vars for string "true"
|
|
Then the interpolated result should be boolean True
|
|
|
|
|
|
Scenario: _interpolate_env_vars converts integer strings
|
|
When I interpolate env vars for string "42"
|
|
Then the interpolated result should be integer 42
|
|
|
|
|
|
Scenario: _interpolate_env_vars converts float strings
|
|
When I interpolate env vars for string "3.14"
|
|
Then the interpolated result should be float 3.14
|
|
|
|
|
|
Scenario: _interpolate_env_vars handles boolean defaults
|
|
When I interpolate env vars with "${MISSING_BOOL:true}"
|
|
Then the interpolated result should be string "True"
|
|
|
|
|
|
Scenario: _interpolate_env_vars handles integer defaults
|
|
When I interpolate env vars with "${MISSING_INT:99}"
|
|
Then the interpolated result should be string "99"
|
|
|
|
|
|
Scenario: _interpolate_env_vars recurses into dicts and lists
|
|
When I interpolate env vars in a nested structure
|
|
Then all nested string values should be interpolated
|
|
|
|
|
|
Scenario: from_blob extracts provider and model
|
|
When I create an ActorConfiguration from a blob with provider and model
|
|
Then the config provider should be set
|
|
And the config model should be set
|
|
|
|
|
|
Scenario: from_blob raises when provider missing
|
|
When I create an ActorConfiguration from a blob without provider
|
|
Then an actor config ValueError should mention "provider"
|
|
|
|
|
|
Scenario: from_blob raises when model missing
|
|
When I create an ActorConfiguration from a blob without model
|
|
Then an actor config ValueError should mention "model"
|
|
|
|
|
|
Scenario: from_blob merges option overrides
|
|
When I create an ActorConfiguration from a blob with option overrides
|
|
Then the config options should include the overrides
|
|
|
|
|
|
Scenario: from_file loads and parses config from disk
|
|
Given a temporary JSON config file with provider and model
|
|
When I create an ActorConfiguration from the file
|
|
Then the config provider should be set
|
|
And the config model should be set
|
|
|
|
|
|
# ============================================================
|
|
# Originally from: actor_hierarchy.feature
|
|
# Feature: Hierarchical Actor YAML Schema Extensions
|
|
# ============================================================
|
|
|
|
Scenario: Node with explicit LSP binding
|
|
Given a hierarchy actor YAML with node lsp_binding
|
|
When I parse the hierarchy actor YAML
|
|
Then the hierarchy actor should be valid
|
|
And the hierarchy graph node "code_writer" should have lsp_binding
|
|
And the hierarchy node "code_writer" lsp_binding server should be "local/pyright"
|
|
And the hierarchy node "code_writer" lsp_binding languages should contain "python"
|
|
|
|
|
|
Scenario: Node with auto LSP binding
|
|
Given a hierarchy actor YAML with auto lsp_binding
|
|
When I parse the hierarchy actor YAML
|
|
Then the hierarchy actor should be valid
|
|
And the hierarchy graph node "implementer" should have lsp_binding
|
|
And the hierarchy node "implementer" lsp_binding auto should be true
|
|
|
|
|
|
Scenario: Node without LSP binding is valid
|
|
Given a hierarchy actor YAML with no lsp_binding
|
|
When I parse the hierarchy actor YAML
|
|
Then the hierarchy actor should be valid
|
|
And the hierarchy graph node "agent_node" should not have lsp_binding
|
|
|
|
|
|
Scenario: LSP binding with invalid server name
|
|
Given a hierarchy actor YAML with lsp_binding server "no-namespace"
|
|
When I parse the hierarchy actor YAML expecting failure
|
|
Then the hierarchy parse error should mention "lsp_binding"
|
|
|
|
# -------------------------------------------------------------------
|
|
# Tool-Source References on NodeDefinition
|
|
# -------------------------------------------------------------------
|
|
|
|
|
|
Scenario: Node with tool_sources referencing skills
|
|
Given a hierarchy actor YAML with node tool_sources skills
|
|
When I parse the hierarchy actor YAML
|
|
Then the hierarchy actor should be valid
|
|
And the hierarchy graph node "executor" should have tool_sources
|
|
And the hierarchy node "executor" tool_sources should have 1 items
|
|
|
|
|
|
Scenario: Node with tool_sources referencing MCP server
|
|
Given a hierarchy actor YAML with node tool_sources mcp
|
|
When I parse the hierarchy actor YAML
|
|
Then the hierarchy actor should be valid
|
|
And the hierarchy node "executor" tool_sources item 0 type should be "mcp"
|
|
And the hierarchy node "executor" tool_sources item 0 name should be "local/filesystem"
|
|
|
|
|
|
Scenario: Node with tool_sources referencing builtin group
|
|
Given a hierarchy actor YAML with node tool_sources builtin
|
|
When I parse the hierarchy actor YAML
|
|
Then the hierarchy actor should be valid
|
|
And the hierarchy node "executor" tool_sources item 0 type should be "builtin"
|
|
And the hierarchy node "executor" tool_sources item 0 group should be "file_operations"
|
|
|
|
|
|
Scenario: Node with mixed tool_sources
|
|
Given a hierarchy actor YAML with mixed node tool_sources
|
|
When I parse the hierarchy actor YAML
|
|
Then the hierarchy actor should be valid
|
|
And the hierarchy node "executor" tool_sources should have 3 items
|
|
|
|
|
|
Scenario: Node with invalid tool_sources type
|
|
Given a hierarchy actor YAML with invalid tool_sources type
|
|
When I parse the hierarchy actor YAML expecting failure
|
|
Then the hierarchy parse error should mention "tool_sources"
|
|
|
|
|
|
Scenario: Node without tool_sources is valid
|
|
Given a hierarchy actor YAML with no lsp_binding
|
|
When I parse the hierarchy actor YAML
|
|
Then the hierarchy actor should be valid
|
|
And the hierarchy graph node "agent_node" should not have tool_sources
|
|
|
|
# -------------------------------------------------------------------
|
|
# Subgraph Node Enhancements
|
|
# -------------------------------------------------------------------
|
|
|
|
|
|
Scenario: Subgraph node with actor_ref
|
|
Given a hierarchy actor YAML with subgraph actor_ref
|
|
When I parse the hierarchy actor YAML
|
|
Then the hierarchy actor should be valid
|
|
And the hierarchy graph node "review" should have type "subgraph"
|
|
And the hierarchy node "review" actor_ref should be "local/code-reviewer"
|
|
|
|
|
|
Scenario: Subgraph node with actor_ref non-namespaced fails
|
|
Given a hierarchy actor YAML with subgraph actor_ref "bad-ref"
|
|
When I parse the hierarchy actor YAML expecting failure
|
|
Then the hierarchy parse error should mention "actor_ref"
|
|
|
|
# -------------------------------------------------------------------
|
|
# Skills Field on ActorConfigSchema
|
|
# -------------------------------------------------------------------
|
|
|
|
|
|
Scenario: Actor with skills list
|
|
Given a hierarchy actor YAML with skills list
|
|
When I parse the hierarchy actor YAML
|
|
Then the hierarchy actor should be valid
|
|
And the hierarchy actor should have 2 skills
|
|
|
|
|
|
Scenario: Actor with empty skills is valid
|
|
Given a hierarchy actor YAML with empty skills
|
|
When I parse the hierarchy actor YAML
|
|
Then the hierarchy actor should be valid
|
|
And the hierarchy actor should have 0 skills
|
|
|
|
# -------------------------------------------------------------------
|
|
# LSP Fields on ActorConfigSchema (top-level)
|
|
# -------------------------------------------------------------------
|
|
|
|
|
|
Scenario: Actor with lsp list
|
|
Given a hierarchy actor YAML with lsp list
|
|
When I parse the hierarchy actor YAML
|
|
Then the hierarchy actor should be valid
|
|
And the hierarchy actor lsp should have 2 items
|
|
|
|
|
|
Scenario: Actor with lsp auto binding
|
|
Given a hierarchy actor YAML with lsp auto
|
|
When I parse the hierarchy actor YAML
|
|
Then the hierarchy actor should be valid
|
|
And the hierarchy actor lsp_auto should be true
|
|
|
|
|
|
Scenario: Actor with lsp_capabilities list
|
|
Given a hierarchy actor YAML with lsp_capabilities
|
|
When I parse the hierarchy actor YAML
|
|
Then the hierarchy actor should be valid
|
|
And the hierarchy actor lsp_capabilities should contain "diagnostics"
|
|
|
|
|
|
Scenario: Actor with lsp_capabilities all
|
|
Given a hierarchy actor YAML with lsp_capabilities all
|
|
When I parse the hierarchy actor YAML
|
|
Then the hierarchy actor should be valid
|
|
And the hierarchy actor lsp_capabilities should be "all"
|
|
|
|
# -------------------------------------------------------------------
|
|
# Precise Validation Error Messages
|
|
# -------------------------------------------------------------------
|
|
|
|
|
|
Scenario: Missing entry node gives precise field path
|
|
Given a hierarchy actor YAML with bad entry_node "nonexistent"
|
|
When I parse the hierarchy actor YAML expecting failure
|
|
Then the hierarchy parse error should mention "entry_node"
|
|
And the hierarchy parse error should mention "nonexistent"
|
|
|
|
|
|
Scenario: Duplicate node IDs gives precise error
|
|
Given a hierarchy actor YAML with duplicate node ids
|
|
When I parse the hierarchy actor YAML expecting failure
|
|
Then the hierarchy parse error should mention "Duplicate"
|
|
|
|
|
|
Scenario: Cycle detection gives node path
|
|
Given a hierarchy actor YAML with cycle
|
|
When I parse the hierarchy actor YAML expecting failure
|
|
Then the hierarchy parse error should mention "cycle"
|
|
|
|
|
|
Scenario: Edge referencing non-existent node gives precise error
|
|
Given a hierarchy actor YAML with bad edge from_node "ghost"
|
|
When I parse the hierarchy actor YAML expecting failure
|
|
Then the hierarchy parse error should mention "ghost"
|
|
|
|
# -------------------------------------------------------------------
|
|
# Loader Integration with Hierarchical Actors
|
|
# -------------------------------------------------------------------
|
|
|
|
|
|
Scenario: Loader discovers hierarchical actor YAML
|
|
Given a hierarchy temp dir with a hierarchical actor YAML file
|
|
When I create a hierarchy loader and run discovery
|
|
Then the hierarchy loader should find 1 actors
|
|
And the hierarchy loaded actor should have skills
|
|
And the hierarchy loaded actor should have lsp
|
|
|
|
|
|
Scenario: Loader rejects actor with invalid lsp_binding
|
|
Given a hierarchy temp dir with invalid lsp_binding actor YAML
|
|
When I create a hierarchy loader and run discovery expecting error
|
|
Then the hierarchy loader error should mention "lsp_binding"
|
|
|
|
|
|
# ============================================================
|
|
# Originally from: actor_registry_coverage.feature
|
|
# Feature: Actor registry coverage
|
|
# ============================================================
|
|
|
|
# ============================================================
|
|
# Originally from: actor_registry_full_coverage.feature
|
|
# Feature: Actor registry full coverage
|
|
# ============================================================
|
|
|
|
Scenario: Registry returns empty list when no providers are configured
|
|
Given a fresh actor registry with no providers
|
|
When I run ensure_built_in_actors
|
|
Then the generated actor list should be empty
|
|
|
|
|
|
Scenario: Registry resolves virtual built-in actors from configured providers in alphabetical order
|
|
Given a fresh actor registry with providers
|
|
| type | name | model |
|
|
| openai | OpenAI | gpt-4o-mini |
|
|
| anthropic | Claude | opus |
|
|
When I run ensure_built_in_actors
|
|
Then the generated actors should be ["claude/opus", "openai/gpt-4o-mini"]
|
|
|
|
|
|
Scenario: Registry resolves virtual built-in actors including preferred provider
|
|
Given preferred provider defaults provider "claude" and model "opus"
|
|
And a fresh actor registry with providers
|
|
| type | name | model |
|
|
| openai | OpenAI | gpt-4o-mini |
|
|
| anthropic | Claude | opus |
|
|
When I run ensure_built_in_actors
|
|
Then the generated actors should be ["claude/opus", "openai/gpt-4o-mini"]
|
|
|
|
|
|
Scenario: Registry virtual resolution does not overwrite pre-existing default preference
|
|
Given preferred provider defaults provider "claude" and model "opus"
|
|
And a fresh actor registry with providers
|
|
| type | name | model |
|
|
| openai | OpenAI | gpt-4o-mini |
|
|
| anthropic | Claude | opus |
|
|
And a pre-existing default actor "OpenAI/gpt-4o-mini"
|
|
When I run ensure_built_in_actors
|
|
Then the service default actor should be "OpenAI/gpt-4o-mini"
|
|
|
|
|
|
Scenario: Built-in actor includes graph descriptor with provider registry source and capabilities
|
|
Given a fresh actor registry with providers
|
|
| type | name | model |
|
|
| openai | OpenAI | gpt-4o-mini |
|
|
When I run ensure_built_in_actors
|
|
Then the built-in actor "openai/gpt-4o-mini" should have graph descriptor with source "provider-registry"
|
|
And the built-in actor "openai/gpt-4o-mini" config blob should include capabilities
|
|
|
|
# ── upsert_actor ────────────────────────────────────────────────────
|
|
|
|
|
|
Scenario: Registry upserts a custom actor with provider model and graph descriptor
|
|
Given a fresh actor registry with no providers
|
|
When I upsert a custom actor "custom/demo" with provider "custom" model "demo" and graph {"flow": "simple"}
|
|
Then the upserted actor should have name "custom/demo"
|
|
And the upserted actor should have provider "custom" and model "demo"
|
|
|
|
|
|
Scenario: Registry rejects unsafe actor configuration without explicit unsafe flag
|
|
Given a fresh actor registry with no providers
|
|
When I attempt to upsert an unsafe actor "local/danger" without allow_unsafe
|
|
Then a ValidationError should be raised with message containing "unsafe"
|
|
|
|
|
|
Scenario: Registry allows unsafe actor when allow_unsafe flag is set
|
|
Given a fresh actor registry with no providers
|
|
When I upsert an unsafe actor "local/allowed" with allow_unsafe flag
|
|
Then the upserted actor should have name "local/allowed"
|
|
And the upserted actor should be marked unsafe
|
|
|
|
|
|
Scenario: Registry allows unsafe actor when allow_unsafe flag is set (is_built_in removed)
|
|
Given a fresh actor registry with no providers
|
|
When I upsert an unsafe actor "local/builtin" with allow_unsafe flag
|
|
Then the upserted actor should have name "local/builtin"
|
|
And the upserted actor should be marked unsafe
|
|
|
|
|
|
Scenario: Registry builds default graph descriptor when provider and model are supplied
|
|
Given a fresh actor registry with no providers
|
|
When I upsert a custom actor "local/graphed" with provider "openai" model "gpt-4o" and no explicit graph
|
|
Then the upserted actor config blob should contain a graph_descriptor with provider "openai" and model "gpt-4o"
|
|
|
|
|
|
Scenario: Registry upsert canonicalizes blob with default options and source
|
|
Given a fresh actor registry with no providers
|
|
When I upsert actor "local/canon" with config blob {"provider": "test", "model": "m1", "foo": "bar"}
|
|
Then the upserted actor config blob should have key "source" with value "custom"
|
|
And the upserted actor config blob should have key "unsafe" with value false
|
|
|
|
# ── Wrapper methods ─────────────────────────────────────────────────
|
|
|
|
|
|
Scenario: Registry get_actor delegates to actor service after ensuring built-ins
|
|
Given a fresh actor registry with providers
|
|
| type | name | model |
|
|
| openai | OpenAI | gpt-4o-mini |
|
|
When I call registry get_actor with "openai/gpt-4o-mini"
|
|
Then the fetched actor via registry should be named "openai/gpt-4o-mini"
|
|
|
|
|
|
Scenario: Registry list_actors returns virtual built-in actors without DB writes
|
|
Given a fresh actor registry with providers
|
|
| type | name | model |
|
|
| openai | OpenAI | gpt-4o-mini |
|
|
| anthropic | Claude | opus |
|
|
When I call registry list_actors
|
|
Then the returned actor list should contain ["claude/opus", "openai/gpt-4o-mini"]
|
|
|
|
|
|
Scenario: Registry remove_actor rejects virtual built-in actors with ValidationError
|
|
Given a fresh actor registry with providers
|
|
| type | name | model |
|
|
| openai | OpenAI | gpt-4o-mini |
|
|
| anthropic | Claude | opus |
|
|
When I call registry remove_actor with "openai/gpt-4o-mini"
|
|
Then the remove_actor call should have raised a ValidationError
|
|
|
|
Scenario: Registry remove_actor delegates to actor service for custom actors
|
|
Given a fresh actor registry with no providers
|
|
When I call registry remove_actor with "local/my-custom-actor"
|
|
Then the service should no longer contain actor "local/my-custom-actor"
|
|
|
|
|
|
Scenario: Registry set_default_actor stores virtual built-in name as preference
|
|
Given a fresh actor registry with providers
|
|
| type | name | model |
|
|
| openai | OpenAI | gpt-4o-mini |
|
|
| anthropic | Claude | opus |
|
|
When I call registry set_default_actor with "claude/opus"
|
|
And I call registry get_default_actor
|
|
Then the returned default actor should be "claude/opus"
|
|
|
|
|
|
Scenario: Registry get_default_actor returns None when no default is configured
|
|
Given a fresh actor registry with providers
|
|
| type | name | model |
|
|
| openai | OpenAI | gpt-4o-mini |
|
|
When I call registry get_default_actor
|
|
Then the returned default actor should be None
|
|
|
|
# ── _canonical_blob ─────────────────────────────────────────────────
|
|
|
|
|
|
Scenario: Canonical blob sets provider model and source defaults from configuration
|
|
Given a fresh actor registry with no providers
|
|
When I upsert actor "local/defaults" with config blob {"provider": "p1", "model": "m1"}
|
|
Then the upserted actor config blob should have key "provider" with value "p1"
|
|
And the upserted actor config blob should have key "model" with value "m1"
|
|
And the upserted actor config blob should have key "source" with value "custom"
|
|
|
|
|
|
Scenario: Canonical blob preserves existing graph descriptor in raw blob
|
|
Given a fresh actor registry with no providers
|
|
When I upsert actor "local/existing" with config blob {"provider": "p1", "model": "m1", "graph_descriptor": {"custom": true}}
|
|
Then the upserted actor config blob graph_descriptor should equal {"custom": true}
|
|
|
|
|
|
Scenario: Canonical blob handles non-dict options by resetting to empty dict
|
|
Given a fresh actor registry with no providers
|
|
When I upsert actor "local/badopts" with config blob {"provider": "p1", "model": "m1", "options": "not-a-dict"}
|
|
Then the upserted actor config blob should have key "options" with value {}
|
|
|
|
# ── _build_graph_descriptor ─────────────────────────────────────────
|
|
|
|
|
|
Scenario: Build graph descriptor includes provider model source and capabilities
|
|
Given a fresh actor registry with no providers
|
|
When I call _build_graph_descriptor with provider "openai" model "gpt-4o" source "test" and capabilities
|
|
Then the graph descriptor should have provider "openai" model "gpt-4o" source "test"
|
|
And the graph descriptor capabilities should not be None
|
|
|
|
|
|
Scenario: Build graph descriptor omits capabilities when not provided
|
|
Given a fresh actor registry with no providers
|
|
When I call _build_graph_descriptor with provider "openai" model "gpt-4o" source "test" and no capabilities
|
|
Then the graph descriptor should have provider "openai" model "gpt-4o" source "test"
|
|
And the graph descriptor capabilities should be None
|
|
|
|
|
|
# ============================================================
|
|
# Originally from: actor_registry_persistence.feature
|
|
# Feature: Actor registry persistence with YAML text retention
|
|
# ============================================================
|
|
|
|
Scenario: Add actor from YAML text retains original source
|
|
Given an actor registry with no configured providers for persistence
|
|
When I add actor from YAML text:
|
|
"""
|
|
name: local/test-actor
|
|
type: llm
|
|
description: A test actor
|
|
provider: openai
|
|
model: gpt-4
|
|
"""
|
|
Then the actor "local/test-actor" should have yaml_text containing "provider: openai"
|
|
And the actor "local/test-actor" should have schema_version "1.0"
|
|
|
|
|
|
Scenario: Add actor with explicit schema version
|
|
Given an actor registry with no configured providers for persistence
|
|
When I add actor from YAML text with schema_version "2.0":
|
|
"""
|
|
name: local/versioned
|
|
type: llm
|
|
description: A versioned actor
|
|
provider: anthropic
|
|
model: claude-3
|
|
"""
|
|
Then the actor "local/versioned" should have schema_version "2.0"
|
|
|
|
|
|
Scenario: Add actor with compiled metadata
|
|
Given an actor registry with no configured providers for persistence
|
|
When I add actor from YAML text with compiled_metadata:
|
|
"""
|
|
name: local/compiled
|
|
type: llm
|
|
description: A compiled actor
|
|
provider: openai
|
|
model: gpt-4o
|
|
"""
|
|
Then the actor "local/compiled" should have compiled_metadata key "graph_nodes"
|
|
|
|
|
|
Scenario: Update existing actor preserves YAML text
|
|
Given an actor registry with no configured providers for persistence
|
|
When I add actor from YAML text:
|
|
"""
|
|
name: local/updatable
|
|
type: llm
|
|
description: An updatable actor
|
|
provider: openai
|
|
model: gpt-4
|
|
"""
|
|
And I update actor from YAML text:
|
|
"""
|
|
name: local/updatable
|
|
type: llm
|
|
description: An updatable actor
|
|
provider: openai
|
|
model: gpt-4-turbo
|
|
"""
|
|
Then the actor "local/updatable" should have yaml_text containing "gpt-4-turbo"
|
|
|
|
|
|
Scenario: Remove actor by namespaced name
|
|
Given an actor registry with no configured providers for persistence
|
|
When I add actor from YAML text:
|
|
"""
|
|
name: local/removable
|
|
type: llm
|
|
description: A removable actor
|
|
provider: openai
|
|
model: gpt-4
|
|
"""
|
|
And I remove actor "local/removable" via registry
|
|
Then the actor "local/removable" should not exist
|
|
|
|
|
|
Scenario: List actors with namespace filter
|
|
Given an actor registry with no configured providers for persistence
|
|
When I add actor from YAML text:
|
|
"""
|
|
name: local/alpha
|
|
type: llm
|
|
description: An alpha actor
|
|
provider: openai
|
|
model: gpt-4
|
|
"""
|
|
And I add actor from YAML text with update:
|
|
"""
|
|
name: local/beta
|
|
type: llm
|
|
description: A beta actor
|
|
provider: anthropic
|
|
model: claude-3
|
|
"""
|
|
Then listing actors with namespace "local" should return 2 actors
|
|
And listing actors with namespace "nonexistent" should return 0 actors
|
|
|
|
|
|
Scenario: Namespaced name auto-prefixing
|
|
Given an actor registry with no configured providers for persistence
|
|
When I add actor from YAML text:
|
|
"""
|
|
name: local/auto-prefixed
|
|
type: llm
|
|
description: An auto-prefixed actor
|
|
provider: openai
|
|
model: gpt-4
|
|
"""
|
|
Then the actor "local/auto-prefixed" should exist
|
|
|
|
|
|
Scenario: Get actor by namespaced name
|
|
Given an actor registry with no configured providers for persistence
|
|
When I add actor from YAML text:
|
|
"""
|
|
name: local/fetchable
|
|
type: llm
|
|
description: A fetchable actor
|
|
provider: openai
|
|
model: gpt-4
|
|
"""
|
|
Then getting actor "local/fetchable" via registry should return provider "openai"
|
|
|
|
|
|
Scenario: Add actor without name raises validation error
|
|
Given an actor registry with no configured providers for persistence
|
|
When I attempt to add actor from YAML text without name:
|
|
"""
|
|
type: llm
|
|
description: No name actor
|
|
provider: openai
|
|
model: gpt-4
|
|
"""
|
|
Then a validation error should be raised for missing name
|
|
|
|
|
|
Scenario: Add duplicate actor without update raises validation error
|
|
Given an actor registry with no configured providers for persistence
|
|
When I add actor from YAML text:
|
|
"""
|
|
name: local/duplicate
|
|
type: llm
|
|
description: A duplicate actor
|
|
provider: openai
|
|
model: gpt-4
|
|
"""
|
|
And I attempt to add duplicate actor from YAML text:
|
|
"""
|
|
name: local/duplicate
|
|
type: llm
|
|
description: A duplicate actor
|
|
provider: openai
|
|
model: gpt-4-turbo
|
|
"""
|
|
Then a validation error should be raised for duplicate actor
|
|
|
|
|
|
Scenario: Upsert actor via legacy API with yaml_text and schema_version
|
|
Given an actor registry with no configured providers for persistence
|
|
When I upsert actor "local/legacy-yaml" with yaml_text and schema_version "1.5"
|
|
Then the actor "local/legacy-yaml" should have yaml_text containing "legacy yaml content"
|
|
And the actor "local/legacy-yaml" should have schema_version "1.5"
|
|
|
|
|
|
|
|
# ============================================================
|
|
# Originally from: actor_runtime.feature
|
|
# Feature: Tool-Calling Actor Runtime
|
|
# ============================================================
|
|
|
|
Scenario: Tool-call loop with single tool call
|
|
Given a tool registry with an echo tool
|
|
And a tool runner for the actor registry
|
|
And a mock LLM caller that requests one tool call then responds
|
|
And a tool-calling runtime with the mock LLM caller
|
|
When I run the tool loop with prompt "Use the echo tool"
|
|
Then the run result content should be "Final answer after tool call"
|
|
And the run result should have 1 tool call records
|
|
And the run result iterations should be 2
|
|
And the run result should not be terminated by limit
|
|
|
|
# ---- Multiple Sequential Tool Calls ----
|
|
|
|
|
|
Scenario: Tool-call loop with multiple sequential tool calls
|
|
Given a tool registry with an echo tool
|
|
And a tool registry with an adder tool
|
|
And a tool runner for the actor registry
|
|
And a mock LLM caller that requests two sequential tool calls then responds
|
|
And a tool-calling runtime with the mock LLM caller
|
|
When I run the tool loop with prompt "Use both tools"
|
|
Then the run result content should be "Done after two tool calls"
|
|
And the run result should have 2 tool call records
|
|
And the run result iterations should be 3
|
|
And the run result should not be terminated by limit
|
|
|
|
# ---- Max Iteration Safety ----
|
|
|
|
|
|
Scenario: Max-iteration safety termination
|
|
Given a tool registry with an echo tool
|
|
And a tool runner for the actor registry
|
|
And a mock LLM caller that always requests tool calls
|
|
And a tool-calling runtime with max iterations 3
|
|
When I run the tool loop with prompt "Keep calling tools"
|
|
Then the run result should be terminated by limit
|
|
And the run result iterations should be 3
|
|
And the run result should have 3 tool call records
|
|
|
|
# ---- Tool Not Found Error ----
|
|
|
|
|
|
Scenario: Tool call error handling for tool not found
|
|
Given a tool registry with an echo tool
|
|
And a tool runner for the actor registry
|
|
And a mock LLM caller that requests a nonexistent tool then responds
|
|
And a tool-calling runtime with the mock LLM caller
|
|
When I run the tool loop with prompt "Call missing tool"
|
|
Then the run result should have 1 tool call records
|
|
And the first tool call record should have success False
|
|
And the first tool call record should have an error message
|
|
|
|
# ---- Tool Execution Error ----
|
|
|
|
|
|
Scenario: Tool call error handling for execution error
|
|
Given a tool registry with a failing tool
|
|
And a tool runner for the actor registry
|
|
And a mock LLM caller that requests the failing tool then responds
|
|
And a tool-calling runtime with the mock LLM caller
|
|
When I run the tool loop with prompt "Call failing tool"
|
|
Then the run result should have 1 tool call records
|
|
And the first tool call record should have success False
|
|
And the first tool call record error should contain "RuntimeError"
|
|
|
|
# ---- Metadata Capture ----
|
|
|
|
|
|
Scenario: Tool metadata capture verification
|
|
Given a tool registry with an echo tool
|
|
And a tool runner for the actor registry
|
|
And a mock LLM caller that requests one tool call then responds
|
|
And a tool-calling runtime with the mock LLM caller
|
|
When I run the tool loop with prompt "Echo something"
|
|
Then the first tool call record tool name should be "test/echo"
|
|
And the first tool call record inputs should not be empty
|
|
And the first tool call record output should not be empty
|
|
And the first tool call record duration should be greater than 0
|
|
And the first tool call record iteration should be 1
|
|
|
|
# ---- Empty Tool List ----
|
|
|
|
|
|
Scenario: Empty tool list handling
|
|
Given an empty actor tool registry
|
|
And a tool runner for the actor registry
|
|
And a mock LLM caller that responds immediately without tool calls
|
|
And a tool-calling runtime with the mock LLM caller
|
|
When I run the tool loop with prompt "No tools available"
|
|
Then the run result content should be "No tools needed"
|
|
And the run result should have 0 tool call records
|
|
And the run result iterations should be 1
|
|
|
|
# ---- Sandbox Root Threading ----
|
|
|
|
|
|
Scenario: Sandbox root threading into tool inputs
|
|
Given a tool registry with an input-capturing tool
|
|
And a tool runner for the actor registry
|
|
And a mock LLM caller that requests the capturing tool then responds
|
|
And a tool-calling runtime with the mock LLM caller
|
|
And an actor context with sandbox root "/tmp/sandbox"
|
|
When I run the tool loop with prompt "Use sandbox" and the actor context
|
|
Then the captured tool inputs should contain sandbox root "/tmp/sandbox"
|
|
|
|
# ---- Resource Binding Threading ----
|
|
|
|
|
|
Scenario: Resource bindings threading into tool inputs
|
|
Given a tool registry with an input-capturing tool
|
|
And a tool runner for the actor registry
|
|
And a mock LLM caller that requests the capturing tool then responds
|
|
And a tool-calling runtime with the mock LLM caller
|
|
And an actor context with resource bindings
|
|
When I run the tool loop with prompt "Use resources" and the actor context
|
|
Then the captured tool inputs should contain resource bindings
|
|
|
|
# ---- Constructor Validation ----
|
|
|
|
|
|
Scenario: Runtime rejects invalid max iterations
|
|
Given a tool registry with an echo tool
|
|
And a tool runner for the actor registry
|
|
And a mock LLM caller that responds immediately without tool calls
|
|
Then creating a runtime with max iterations 0 should raise ValueError
|
|
|
|
|
|
Scenario: Runtime rejects empty prompt
|
|
Given a tool registry with an echo tool
|
|
And a tool runner for the actor registry
|
|
And a mock LLM caller that responds immediately without tool calls
|
|
And a tool-calling runtime with the mock LLM caller
|
|
Then running the tool loop with empty prompt should raise ValueError
|
|
|
|
# ---- Actor Context ----
|
|
|
|
|
|
Scenario: ToolActorContext construction and properties
|
|
Given an actor context with plan "plan-abc" phase "execute"
|
|
Then the actor context plan id should be "plan-abc"
|
|
And the actor context phase should be "execute"
|
|
And the actor context tool call history should be empty
|
|
|
|
|
|
Scenario: ToolActorContext records tool calls
|
|
Given an actor context with plan "plan-rec"
|
|
When I record a tool call with name "test/echo" and success True
|
|
Then the actor context tool call history should have 1 records
|
|
|
|
|
|
Scenario: ToolActorContext rejects empty plan id
|
|
Then creating a tool actor context with empty plan id should raise ValueError
|
|
|
|
|
|
Scenario: ToolActorContext summary includes metadata
|
|
Given an actor context with plan "plan-sum" sandbox "/tmp/s"
|
|
Then the actor context summary should contain plan id "plan-sum"
|
|
And the actor context summary should contain sandbox root "/tmp/s"
|
|
|
|
# ---- Router Integration ----
|
|
|
|
|
|
Scenario: Tool-call loop with router integration
|
|
Given a tool registry with an echo tool
|
|
And a tool runner for the actor registry
|
|
And a tool call router with plan id "plan-r"
|
|
And a mock LLM caller that requests one tool call then responds
|
|
And a tool-calling runtime with the mock LLM caller and router
|
|
When I run the tool loop with prompt "Use echo with router"
|
|
Then the run result content should be "Final answer after tool call"
|
|
And the run result should have 1 tool call records
|
|
|
|
# ---- ToolCallRecord Model ----
|
|
|
|
|
|
Scenario: ToolCallRecord model fields
|
|
Given a tool call record with name "test/tool" and duration 42.5
|
|
Then the tool call record tool name should be "test/tool"
|
|
And the tool call record duration should be 42.5
|
|
|
|
# ---- Default Context ----
|
|
|
|
|
|
Scenario: Tool loop creates default context when none provided
|
|
Given a tool registry with an echo tool
|
|
And a tool runner for the actor registry
|
|
And a mock LLM caller that responds immediately without tool calls
|
|
And a tool-calling runtime with the mock LLM caller
|
|
When I run the tool loop with prompt "No context" and no context
|
|
Then the run result content should be "No tools needed"
|
|
And the run result should have 0 tool call records
|
|
|
|
# ---- Actor Context Additional Properties ----
|
|
|
|
|
|
Scenario: ToolActorContext exposes all property accessors
|
|
Given an actor context with all optional fields
|
|
Then the actor context automation profile should be "auto-profile"
|
|
And the actor context project resources should have key "repo"
|
|
And the actor context metadata should have key "version"
|
|
And the actor context clear history should empty the history
|
|
|
|
|
|
Scenario: ToolActorContext rejects non-ToolCallRecord in record_tool_call
|
|
Given an actor context with plan "plan-typecheck"
|
|
Then recording a non-ToolCallRecord should raise TypeError
|
|
|
|
# ---- Runtime Constructor Type Validation ----
|
|
|
|
|
|
Scenario: Runtime rejects invalid registry type
|
|
Then creating a runtime with a non-ToolRegistry should raise TypeError
|
|
|
|
|
|
Scenario: Runtime rejects invalid runner type
|
|
Given a tool registry with an echo tool
|
|
Then creating a runtime with a non-ToolRunner should raise TypeError
|
|
|
|
# ---- Runtime Property Accessors ----
|
|
|
|
|
|
Scenario: Runtime exposes max_iterations and provider_format properties
|
|
Given a tool registry with an echo tool
|
|
And a tool runner for the actor registry
|
|
And a mock LLM caller that responds immediately without tool calls
|
|
And a tool-calling runtime with max iterations 5
|
|
Then the runtime max iterations should be 5
|
|
And the runtime provider format should not be None
|
|
|
|
# ---- Generic Exception in Tool Execution ----
|
|
|
|
|
|
Scenario: Tool call handles unexpected generic exception
|
|
Given a tool registry with a tool that raises a generic exception
|
|
And a tool runner for the actor registry
|
|
And a mock LLM caller that requests the generic-error tool then responds
|
|
And a tool-calling runtime with the mock LLM caller
|
|
When I run the tool loop with prompt "Trigger generic error"
|
|
Then the run result should have 1 tool call records
|
|
And the first tool call record should have success False
|
|
And the first tool call record error should contain "KeyError"
|
|
|
|
|
|
# ============================================================
|
|
# Originally from: m2_actor_tool_smoke.feature
|
|
# Feature: M2 actor + tool source smoke suite
|
|
# ============================================================
|
|
|
|
Scenario: Load hierarchical actor from YAML fixture
|
|
Given I load the M2 hierarchical actor fixture
|
|
Then the M2 actor name should be "m2test/hierarchical-workflow"
|
|
And the M2 actor type should be "graph"
|
|
And the M2 actor should have 3 graph nodes
|
|
And the M2 actor entry node should be "planner"
|
|
And the M2 actor exit nodes should include "reviewer"
|
|
|
|
|
|
Scenario: Load hierarchical actor via ActorLoader from fixture directory
|
|
Given I create an M2 actor loader from the fixture directory
|
|
When I run M2 actor discovery
|
|
Then the M2 loader should find 1 actors
|
|
And the M2 loader should contain actor "m2test/hierarchical-workflow"
|
|
|
|
# ── Skill pack loading and registry ──────────────────────────────
|
|
|
|
|
|
Scenario: Load skill pack from YAML fixture
|
|
Given I load the M2 skill pack fixture
|
|
Then the M2 skill name should be "m2test/file-ops-pack"
|
|
And the M2 skill should have 2 tool references
|
|
And the M2 skill should have 1 inline tools
|
|
|
|
|
|
Scenario: Register skill pack and verify tool resolution
|
|
Given I register the M2 skill pack in a skill registry
|
|
When I resolve tools for M2 skill "m2test/file-ops-pack"
|
|
Then the M2 resolved tools should include a tool_ref entry
|
|
And the M2 resolved tools should include an inline entry
|
|
|
|
|
|
Scenario: Skill registry lists registered skills correctly
|
|
Given I register the M2 skill pack in a skill registry
|
|
When I list all M2 skills in the registry
|
|
Then the M2 skill list should contain "m2test/file-ops-pack"
|
|
And the M2 skill list should have 1 entries
|
|
|
|
|
|
Scenario: Invoke built-in tool through skill context
|
|
Given I create an M2 skill context with sandbox
|
|
When I register a tool invocation for "builtin/read_file" in the M2 context
|
|
Then the M2 skill context should have 1 invocation records
|
|
|
|
# ── Tool lifecycle ───────────────────────────────────────────────
|
|
|
|
|
|
Scenario: Tool lifecycle discover activate execute deactivate
|
|
Given I create an M2 tool runtime with a mock tool "m2test/echo"
|
|
And I create an M2 execution context with plan "m2-plan-001"
|
|
When I discover M2 tool "m2test/echo"
|
|
Then the M2 discovered descriptor name should be "m2test/echo"
|
|
When I activate M2 tool "m2test/echo"
|
|
And I execute M2 tool "m2test/echo" with params text "hello"
|
|
Then the M2 execution result should be successful
|
|
When I deactivate M2 tool "m2test/echo"
|
|
Then the M2 tool should be deactivated
|
|
|
|
|
|
Scenario: Tool registry registers and retrieves tools
|
|
Given I create an M2 tool registry
|
|
When I register an M2 tool spec "m2test/file-reader"
|
|
Then getting M2 tool "m2test/file-reader" should return a spec
|
|
And listing M2 tools should include "m2test/file-reader"
|
|
|
|
# ── MCP stub tool discovery and invocation ───────────────────────
|
|
|
|
|
|
Scenario: MCP stub tool discovery and invocation
|
|
Given I start the M2 MCP stub server
|
|
When I discover tools from the M2 MCP stub
|
|
Then the M2 stub should expose 3 tools
|
|
And the M2 stub tools should include "mcp/search"
|
|
And the M2 stub tools should include "mcp/fetch"
|
|
And the M2 stub tools should include "mcp/transform"
|
|
When I invoke M2 MCP stub tool "mcp/search" with query "test"
|
|
Then the M2 MCP stub result should contain results
|
|
And I stop the M2 MCP stub server
|
|
|
|
|
|
Scenario: MCP stub server lifecycle guards
|
|
Given I have an M2 MCP stub server that is stopped
|
|
When I try to discover tools from the stopped M2 stub
|
|
Then an M2 RuntimeError should be raised
|
|
When I start and invoke M2 MCP stub tool "mcp/fetch" with url "http://example.com"
|
|
Then the M2 MCP stub fetch result should have status 200
|
|
And the M2 MCP stub invocation log should have 1 entries
|