HAL9000
9e3bf30bca
CI / lint (pull_request) Successful in 1m2s
CI / helm (pull_request) Successful in 1m2s
CI / build (pull_request) Successful in 1m10s
CI / typecheck (pull_request) Successful in 1m19s
CI / security (pull_request) Successful in 1m20s
CI / quality (pull_request) Successful in 1m39s
CI / push-validation (pull_request) Successful in 44s
CI / unit_tests (pull_request) Successful in 10m27s
CI / docker (pull_request) Successful in 2m48s
CI / integration_tests (pull_request) Successful in 17m20s
CI / coverage (pull_request) Successful in 22m15s
CI / status-check (pull_request) Successful in 4s
Three issues causing CI failures in advanced-context-strategies tests:
1. AmbiguousStep: `@then("the strategy should be {strategy_type}")` in
advanced_context_strategies_steps.py conflicted with the existing
`@then('the strategy should be "{expected_strategy}"')` in
plan_merge_strategy_steps.py:122. Renamed to
`@then("the loaded strategy type should be {strategy_type}")` and
updated all four matching lines in the feature file.
2. Wrong fragment count assertion: scenario "Semantic search strategy
ranks by embedding similarity" expected 3 fragments but
SemanticEmbeddingStrategy (word-overlap Jaccard, min_similarity=0.05)
correctly filters "File input output handler" (0 overlap with
"database connection"). Fixed assertion from 3 to 2.
3. Robot helper import failure: `features.mocks` is not importable when
Robot Framework imports the library because it adds robot/ to
sys.path but not the project root. Added explicit project-root
sys.path.insert before the features.mocks import (same pattern as
helper_lsp_stub.py), with # noqa: E402 on the post-path imports.
ISSUES CLOSED: #7574
278 lines
14 KiB
Gherkin
278 lines
14 KiB
Gherkin
@phase3 @acms @advanced_context_strategies
|
|
Feature: Advanced Context Strategies Integration Tests
|
|
As a CleverAgents developer
|
|
I want advanced context strategies for semantic search, relevance scoring, and adaptive selection
|
|
So that the ACMS pipeline can intelligently select and combine strategies
|
|
|
|
# ===========================================================================
|
|
# Semantic Search Strategy (with FakeEmbeddings)
|
|
# ===========================================================================
|
|
|
|
@semantic_search
|
|
Scenario: Semantic search strategy ranks by embedding similarity
|
|
Given a semantic search strategy with FakeEmbeddings
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/db.py | Database connection pool manager | 0.5 | 20 | 3 |
|
|
| project://app/io.py | File input output handler | 0.8 | 15 | 3 |
|
|
| project://app/sql.py | SQL database query executor | 0.6 | 25 | 3 |
|
|
And a context budget with max_tokens 1000 and reserved_tokens 0
|
|
When I search with query "database connection"
|
|
Then the first result should have uko_node "project://app/db.py"
|
|
And the result should have 2 fragments
|
|
|
|
@semantic_search
|
|
Scenario: Semantic search filters low-similarity results
|
|
Given a semantic search strategy with FakeEmbeddings
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/db.py | database handler | 0.9 | 20 | 3 |
|
|
| project://app/io.py | file io module | 0.8 | 15 | 3 |
|
|
And a context budget with max_tokens 1000 and reserved_tokens 0
|
|
When I search with query "quantum computing"
|
|
Then 0 fragments should be returned
|
|
|
|
@semantic_search
|
|
Scenario: Semantic search respects token budget
|
|
Given a semantic search strategy with FakeEmbeddings
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | database | 0.9 | 100 | 3 |
|
|
| project://app/b.py | database | 0.8 | 100 | 3 |
|
|
| project://app/c.py | database | 0.7 | 100 | 3 |
|
|
And a context budget with max_tokens 250 and reserved_tokens 0
|
|
When I search with query "database"
|
|
Then 2 fragments should be returned
|
|
|
|
# ===========================================================================
|
|
# Relevance Scoring Strategy
|
|
# ===========================================================================
|
|
|
|
@relevance_scoring
|
|
Scenario: Relevance scoring strategy ranks by relevance score
|
|
Given a relevance scoring strategy
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | alpha | 0.3 | 10 | 3 |
|
|
| project://app/b.py | beta | 0.9 | 10 | 3 |
|
|
| project://app/c.py | gamma | 0.6 | 10 | 3 |
|
|
And a context budget with max_tokens 1000 and reserved_tokens 0
|
|
When I assemble with relevance scoring
|
|
Then the first result should have uko_node "project://app/b.py"
|
|
And the second result should have uko_node "project://app/c.py"
|
|
And the third result should have uko_node "project://app/a.py"
|
|
|
|
@relevance_scoring
|
|
Scenario: Relevance scoring respects budget
|
|
Given a relevance scoring strategy
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | alpha | 0.9 | 100 | 3 |
|
|
| project://app/b.py | beta | 0.8 | 100 | 3 |
|
|
| project://app/c.py | gamma | 0.7 | 100 | 3 |
|
|
And a context budget with max_tokens 250 and reserved_tokens 0
|
|
When I assemble with relevance scoring
|
|
Then 2 fragments should be returned
|
|
|
|
@relevance_scoring
|
|
Scenario: Relevance scoring handles empty input
|
|
Given a relevance scoring strategy
|
|
And an empty context fragment list
|
|
And a context budget with max_tokens 1000 and reserved_tokens 0
|
|
When I assemble with relevance scoring
|
|
Then 0 fragments should be returned
|
|
|
|
# ===========================================================================
|
|
# Adaptive Context Strategy Selector
|
|
# ===========================================================================
|
|
|
|
@adaptive_selector
|
|
Scenario: Adaptive selector chooses best strategy for query
|
|
Given an adaptive context strategy selector
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/db.py | Database connection pool manager | 0.5 | 20 | 3 |
|
|
| project://app/io.py | File input output handler | 0.8 | 15 | 3 |
|
|
| project://app/sql.py | SQL database query executor | 0.6 | 25 | 3 |
|
|
And a context budget with max_tokens 1000 and reserved_tokens 0
|
|
When I select strategy for query "database connection"
|
|
Then the selected strategy should be "semantic-embedding"
|
|
|
|
@adaptive_selector
|
|
Scenario: Adaptive selector falls back to relevance for no query
|
|
Given an adaptive context strategy selector
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | alpha | 0.3 | 10 | 3 |
|
|
| project://app/b.py | beta | 0.9 | 10 | 3 |
|
|
And a context budget with max_tokens 1000 and reserved_tokens 0
|
|
When I select strategy without query
|
|
Then the selected strategy should be "relevance-scoring"
|
|
|
|
@adaptive_selector
|
|
Scenario: Adaptive selector chooses graph navigation for focus nodes
|
|
Given an adaptive context strategy selector
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/io.py | io module | 0.5 | 20 | 5 |
|
|
| project://app/main.py | main entry | 0.9 | 15 | 3 |
|
|
| project://other/lib.py | library | 0.7 | 25 | 9 |
|
|
And a context budget with max_tokens 1000 and reserved_tokens 0
|
|
When I select strategy with focus "project://app"
|
|
Then the selected strategy should be "breadth-depth-navigator"
|
|
|
|
# ===========================================================================
|
|
# Context Fusion Strategy
|
|
# ===========================================================================
|
|
|
|
@context_fusion
|
|
Scenario: Context fusion combines results from multiple strategies
|
|
Given a context fusion strategy with strategies "semantic-embedding,relevance-scoring"
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/db.py | Database connection pool manager | 0.5 | 20 | 3 |
|
|
| project://app/io.py | File input output handler | 0.8 | 15 | 3 |
|
|
| project://app/sql.py | SQL database query executor | 0.6 | 25 | 3 |
|
|
And a context budget with max_tokens 1000 and reserved_tokens 0
|
|
When I fuse with query "database"
|
|
Then at least 2 fragments should be returned by fusion
|
|
And the result should contain fragments from multiple strategies
|
|
|
|
@context_fusion
|
|
Scenario: Context fusion respects budget across strategies
|
|
Given a context fusion strategy with strategies "semantic-embedding,relevance-scoring"
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | database | 0.9 | 100 | 3 |
|
|
| project://app/b.py | database | 0.8 | 100 | 3 |
|
|
| project://app/c.py | database | 0.7 | 100 | 3 |
|
|
And a context budget with max_tokens 250 and reserved_tokens 0
|
|
When I fuse with query "database"
|
|
Then the total tokens should not exceed 250
|
|
|
|
@context_fusion
|
|
Scenario: Context fusion deduplicates results
|
|
Given a context fusion strategy with strategies "semantic-embedding,relevance-scoring"
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | database | 0.9 | 100 | 3 |
|
|
| project://app/b.py | database | 0.8 | 100 | 3 |
|
|
And a context budget with max_tokens 1000 and reserved_tokens 0
|
|
When I fuse with query "database"
|
|
Then each fragment should appear only once in results
|
|
|
|
# ===========================================================================
|
|
# YAML Strategy Configuration
|
|
# ===========================================================================
|
|
|
|
@yaml_config
|
|
Scenario: Load semantic search strategy from YAML
|
|
Given a YAML policy with semantic search configuration
|
|
When I load the strategy from YAML
|
|
Then the loaded strategy type should be "semantic-embedding"
|
|
And the strategy should have min_similarity configured
|
|
|
|
@yaml_config
|
|
Scenario: Load relevance scoring strategy from YAML
|
|
Given a YAML policy with relevance scoring configuration
|
|
When I load the strategy from YAML
|
|
Then the loaded strategy type should be "relevance-scoring"
|
|
|
|
@yaml_config
|
|
Scenario: Load adaptive selector from YAML
|
|
Given a YAML policy with adaptive selector configuration
|
|
When I load the strategy from YAML
|
|
Then the loaded strategy type should be "adaptive-selector"
|
|
And the strategy should have fallback strategy configured
|
|
|
|
@yaml_config
|
|
Scenario: Load context fusion from YAML
|
|
Given a YAML policy with context fusion configuration
|
|
When I load the strategy from YAML
|
|
Then the loaded strategy type should be "context-fusion"
|
|
And the strategy should have multiple strategies configured
|
|
|
|
@yaml_config
|
|
Scenario: YAML configuration with custom parameters
|
|
Given a YAML policy with custom strategy parameters
|
|
When I load the strategy from YAML
|
|
Then the strategy should respect custom parameters
|
|
|
|
# ===========================================================================
|
|
# Integration with ContextAssembler
|
|
# ===========================================================================
|
|
|
|
@integration
|
|
Scenario: Advanced strategies integrate with ContextAssembler
|
|
Given a ContextAssembler with advanced strategies registered
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/db.py | Database connection pool manager | 0.5 | 20 | 3 |
|
|
| project://app/io.py | File input output handler | 0.8 | 15 | 3 |
|
|
| project://app/sql.py | SQL database query executor | 0.6 | 25 | 3 |
|
|
And a context budget with max_tokens 1000 and reserved_tokens 0
|
|
When I assemble context with query "database"
|
|
Then the assembler should select an appropriate strategy
|
|
And the result should be properly ranked
|
|
|
|
@integration
|
|
Scenario: ContextAssembler respects strategy priority
|
|
Given a ContextAssembler with multiple strategies registered
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | alpha | 0.5 | 10 | 3 |
|
|
| project://app/b.py | beta | 0.9 | 10 | 3 |
|
|
And a context budget with max_tokens 1000 and reserved_tokens 0
|
|
When I assemble context with query "test"
|
|
Then the highest-confidence strategy should be selected
|
|
|
|
@integration
|
|
Scenario: ContextAssembler handles strategy fallback
|
|
Given a ContextAssembler with advanced strategies registered
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | alpha | 0.5 | 10 | 3 |
|
|
And a context budget with max_tokens 1000 and reserved_tokens 0
|
|
When I assemble context with unsupported request
|
|
Then the assembler should fall back to default strategy
|
|
|
|
# ===========================================================================
|
|
# Error Handling and Edge Cases
|
|
# ===========================================================================
|
|
|
|
@error_handling
|
|
Scenario: Semantic search handles empty query
|
|
Given a semantic search strategy with FakeEmbeddings
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | alpha | 0.5 | 10 | 3 |
|
|
And a context budget with max_tokens 1000 and reserved_tokens 0
|
|
When I search with empty query
|
|
Then the strategy should fall back to relevance ordering
|
|
|
|
@error_handling
|
|
Scenario: Adaptive selector handles invalid request
|
|
Given an adaptive context strategy selector
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | alpha | 0.5 | 10 | 3 |
|
|
And a context budget with max_tokens 1000 and reserved_tokens 0
|
|
When I select strategy with invalid request
|
|
Then the selector should return a valid strategy
|
|
|
|
@error_handling
|
|
Scenario: Context fusion handles strategy failure
|
|
Given a context fusion strategy with strategies "semantic-embedding,relevance-scoring"
|
|
And the following context fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | alpha | 0.5 | 10 | 3 |
|
|
And a context budget with max_tokens 1000 and reserved_tokens 0
|
|
When one strategy fails during fusion
|
|
Then the fusion should continue with remaining strategies
|
|
|
|
@error_handling
|
|
Scenario: YAML configuration handles missing parameters
|
|
Given a YAML policy with incomplete strategy configuration
|
|
When I load the strategy from YAML
|
|
Then the strategy should use default parameters
|