HAL9000
f7901c404f
Fixes a race condition (TOCTOU) in ActorLoader.list_actors() where the namespace filter was applied outside the threading lock, creating a window for concurrent dictionary modifications. Moving the filter inside the with self._lock: block ensures atomic reads and filtering. - Move namespace filtering inside the RLock in list_actors() - Add concurrency BDD test (threading.Barrier) for list_actors + clear race condition - Update CHANGELOG.md with fix entry (closes #8588) - Update CONTRIBUTORS.md with contribution details ISSUES CLOSED: #8588
272 lines
16 KiB
Gherkin
272 lines
16 KiB
Gherkin
Feature: Actor registry and loader
|
|
As a CleverAgents user
|
|
I want to discover, load, and cache actor YAML configs from the filesystem
|
|
So that actors are available for compilation and execution
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Discovery from search roots
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
Scenario: Discover actors from a single directory
|
|
Given a temporary actors directory with files
|
|
| filename | name | type | model |
|
|
| reviewer.yaml | assistants/reviewer | llm | gpt-4 |
|
|
| formatter.yaml | utilities/formatter | llm | gpt-4 |
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery
|
|
Then the loader should find 2 actors
|
|
And the loader should contain actor "assistants/reviewer"
|
|
And the loader should contain actor "utilities/formatter"
|
|
|
|
Scenario: Discover actors from multiple search roots
|
|
Given a temporary actors directory "root_a" with files
|
|
| filename | name | type | model |
|
|
| writer.yaml | assistants/writer | llm | gpt-4 |
|
|
And a temporary actors directory "root_b" with files
|
|
| filename | name | type | model |
|
|
| analyst.yaml | data/analyst | llm | gpt-4 |
|
|
When I create an actor loader with both directories as search roots
|
|
And I run discovery
|
|
Then the loader should find 2 actors
|
|
And the loader should contain actor "assistants/writer"
|
|
And the loader should contain actor "data/analyst"
|
|
|
|
Scenario: Only YAML files are discovered
|
|
Given a temporary actors directory with files
|
|
| filename | name | type | model |
|
|
| valid.yaml | assistants/valid | llm | gpt-4 |
|
|
And a non-YAML file "readme.txt" in the directory
|
|
And a non-YAML file "config.json" in the directory
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery
|
|
Then the loader should find 1 actors
|
|
|
|
Scenario: Nested YAML files are discovered recursively
|
|
Given a temporary actors directory with nested structure
|
|
| subdir | filename | name | type | model |
|
|
| sub | nested.yaml | assistants/nested | llm | gpt-4 |
|
|
| sub/deep | deep.yaml | assistants/deep | llm | gpt-4 |
|
|
And a top-level actor file
|
|
| filename | name | type | model |
|
|
| top.yaml | assistants/top | llm | gpt-4 |
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery
|
|
Then the loader should find 3 actors
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Duplicate detection
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
Scenario: Duplicate actor names across search roots emit consolidated error
|
|
Given a temporary actors directory "dup_a" with files
|
|
| filename | name | type | model |
|
|
| writer.yaml | assistants/writer | llm | gpt-4 |
|
|
And a temporary actors directory "dup_b" with files
|
|
| filename | name | type | model |
|
|
| writer2.yaml | assistants/writer | llm | gpt-4 |
|
|
When I create an actor loader with both directories as search roots
|
|
And I run discovery expecting errors
|
|
Then the loader should raise a validation error mentioning "assistants/writer"
|
|
|
|
Scenario: Duplicate actor names within a single search root emit error
|
|
Given a temporary actors directory with files
|
|
| filename | name | type | model |
|
|
| one.yaml | assistants/helper | llm | gpt-4 |
|
|
| two.yaml | assistants/helper | llm | gpt-4 |
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery expecting errors
|
|
Then the loader should raise a validation error mentioning "assistants/helper"
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Namespace handling
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
Scenario: Default local/ namespace applied when namespace omitted
|
|
Given a temporary actors directory with a nameless actor file
|
|
| filename | raw_name | type | model |
|
|
| bare.yaml | my-agent | llm | gpt-4 |
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery
|
|
Then the loader should contain actor "local/my-agent"
|
|
|
|
Scenario: Explicit namespace preserved as-is
|
|
Given a temporary actors directory with files
|
|
| filename | name | type | model |
|
|
| custom.yaml | myteam/my-actor | llm | gpt-4 |
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery
|
|
Then the loader should contain actor "myteam/my-actor"
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Namespaced lookup
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
Scenario: Get actor by namespaced name
|
|
Given a temporary actors directory with files
|
|
| filename | name | type | model |
|
|
| helper.yaml | assistants/helper | llm | gpt-4 |
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery
|
|
Then I can retrieve actor "assistants/helper" by name
|
|
And the retrieved actor type should be "llm"
|
|
|
|
Scenario: Get nonexistent actor returns None
|
|
Given a temporary actors directory with files
|
|
| filename | name | type | model |
|
|
| helper.yaml | assistants/helper | llm | gpt-4 |
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery
|
|
Then getting actor "assistants/missing" should return None
|
|
|
|
Scenario: List actors filtered by namespace
|
|
Given a temporary actors directory with files
|
|
| filename | name | type | model |
|
|
| rev.yaml | assistants/reviewer | llm | gpt-4 |
|
|
| fmt.yaml | utilities/formatter | llm | gpt-4 |
|
|
| ana.yaml | assistants/analyzer | llm | gpt-4 |
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery
|
|
Then listing loaded actors with namespace "assistants" should return 2 actors
|
|
And listing loaded actors with namespace "utilities" should return 1 actors
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Content hash caching
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
Scenario: Unchanged files are not reloaded on re-discovery
|
|
Given a temporary actors directory with files
|
|
| filename | name | type | model |
|
|
| cached.yaml | assistants/cached | llm | gpt-4 |
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery
|
|
And I run discovery again
|
|
Then the actor "assistants/cached" should have been loaded only once
|
|
|
|
Scenario: Modified files are reloaded on re-discovery
|
|
Given a temporary actors directory with files
|
|
| filename | name | type | model |
|
|
| mutable.yaml | assistants/mutable | llm | gpt-4 |
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery
|
|
And I modify the actor file "mutable.yaml" to change the model to "gpt-3.5-turbo"
|
|
And I run discovery again
|
|
Then the actor "assistants/mutable" model should be "gpt-3.5-turbo"
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Validation at load time
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
Scenario: Invalid YAML file is rejected with clear error
|
|
Given a temporary actors directory with an invalid YAML file "bad.yaml"
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery expecting errors
|
|
Then the loader should raise a validation error mentioning "bad.yaml"
|
|
|
|
Scenario: YAML file missing required fields is rejected
|
|
Given a temporary actors directory with a schema-invalid file "incomplete.yaml"
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery expecting errors
|
|
Then the loader should raise a validation error
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Integration with example actors
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
Scenario: Discover all example actors from examples/actors/
|
|
Given the project examples/actors directory exists
|
|
When I create an actor loader with the examples/actors directory
|
|
And I run discovery
|
|
Then the loader should find at least 5 actors
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Cache invalidation
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
Scenario: Clear cache removes all loaded actors
|
|
Given a temporary actors directory with files
|
|
| filename | name | type | model |
|
|
| test.yaml | assistants/test | llm | gpt-4 |
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery
|
|
And I clear the loader cache
|
|
Then the loader should find 0 actors
|
|
|
|
Scenario: Deleted actor file is removed on re-discovery
|
|
Given a temporary actors directory with files
|
|
| filename | name | type | model |
|
|
| keep.yaml | assistants/keep | llm | gpt-4 |
|
|
| remove.yaml | assistants/removable | llm | gpt-4 |
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery
|
|
And I delete the actor file "remove.yaml"
|
|
And I run discovery again
|
|
Then the loader should find 1 actors
|
|
And the loader should contain actor "assistants/keep"
|
|
And getting actor "assistants/removable" should return None
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Tool Registry integration
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
Scenario: Actor loader resolves tool references through tool registry
|
|
Given a tool registry with tool "files/read_file"
|
|
And a temporary actors directory with a tool-actor file
|
|
| filename | name | type | tools |
|
|
| tools.yaml | utilities/file_ops | tool | files/read_file |
|
|
When I create an actor loader with that directory and the tool registry
|
|
And I run discovery
|
|
Then the loader should contain actor "utilities/file_ops"
|
|
And the actor "utilities/file_ops" tools should include "files/read_file"
|
|
|
|
Scenario: Actor with unresolvable tool reference logs warning
|
|
Given an empty tool registry
|
|
And a temporary actors directory with a tool-actor file
|
|
| filename | name | type | tools |
|
|
| tools.yaml | utilities/ops | tool | missing/nonexistent |
|
|
When I create an actor loader with that directory and the tool registry
|
|
And I run discovery
|
|
Then the loader should contain actor "utilities/ops"
|
|
And the loader should have a warning about unresolved tool "missing/nonexistent"
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Concurrency and race conditions
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
@unit @actor @concurrency
|
|
Scenario: list_actors with namespace filter is thread-safe under concurrent clear/discover
|
|
Given a temporary actors directory with files
|
|
| filename | name | type | model |
|
|
| rev.yaml | assistants/reviewer | llm | gpt-4 |
|
|
| fmt.yaml | utilities/formatter | llm | gpt-4 |
|
|
| ana.yaml | assistants/analyzer | llm | gpt-4 |
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery
|
|
And 5 threads concurrently call list_actors with namespace "assistants" while another thread calls clear
|
|
Then all concurrent list_actors calls should return consistent results
|
|
|
|
# ────────────────────────────────────────────────────────────
|
|
# Edge cases for coverage
|
|
# ────────────────────────────────────────────────────────────
|
|
|
|
Scenario: Non-existent search root is skipped gracefully
|
|
Given a non-existent directory path as search root
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery
|
|
Then the loader should find 0 actors
|
|
|
|
Scenario: YAML file containing a list instead of a mapping is rejected
|
|
Given a temporary actors directory with a list-content YAML file "list.yaml"
|
|
When I create an actor loader with that directory as search root
|
|
And I run discovery expecting errors
|
|
Then the loader should raise a validation error mentioning "list.yaml"
|
|
|
|
Scenario: LLM actor without tools does not produce warnings with tool registry
|
|
Given a tool registry with tool "files/read_file"
|
|
And a temporary actors directory with files
|
|
| filename | name | type | model |
|
|
| simple.yaml | assistants/simple | llm | gpt-4 |
|
|
When I create an actor loader with that directory and the tool registry
|
|
And I run discovery
|
|
Then the loader should contain actor "assistants/simple"
|
|
And the loader should have 0 warnings
|