HAL9000
df26d166c3
Implemented comprehensive integration tests for advanced context strategies:
Behave Feature File (features/advanced_context_strategies.feature):
- 30+ scenarios covering semantic search, relevance scoring,
adaptive selection, context fusion, YAML config, and integraton
- Uses FakeEmbeddings for deterministic testing without real API calls
Step Definitions (features/steps/advanced_context_strategies_steps.py):
- 50+ step definitions for all test scenarios
- RelevanceScoringStrategy, AdaptiveContextSelector, ContextFusionStrategy
- Full type annotations with pyright compliance
Robot Framework Tests (robot/advanced_context_strategies.robot):
- E2E integration tests for all advanced strategies
- Helper keywords for test execution and strategy creation
Robot Helper (robot/helper_advanced_context_strategies.py):
- Strategy creation/configureation functions
- Fragment and budget management utilities
- Add CHANGELOG.md entry under [Unreleased] section
- Update CONTRIBUTORS.md with contribution entry
ISSUES CLOSED: #7574
280 lines
13 KiB
Plaintext
280 lines
13 KiB
Plaintext
*** Settings ***
|
|
Documentation Advanced Context Strategies Integration Tests
|
|
... Tests for semantic search, relevance scoring, adaptive selection,
|
|
... and context fusion strategies using FakeEmbeddings.
|
|
Library Collections
|
|
Library String
|
|
Library helper_advanced_context_strategies.py
|
|
|
|
*** Test Cases ***
|
|
Semantic Search Strategy Ranks By Similarity
|
|
[Documentation] Verify semantic search ranks fragments by embedding similarity
|
|
[Tags] semantic_search integration
|
|
${strategy}= Create Semantic Search Strategy
|
|
${fragments}= Create Test Fragments
|
|
... project://app/db.py database connection 0.5 20 3
|
|
... project://app/io.py file input output handler 0.8 15 3
|
|
... project://app/sql.py database connection executor 0.6 25 3
|
|
${budget}= Create Context Budget 1000 0
|
|
${results}= Search With Query ${strategy} database connection ${fragments} ${budget}
|
|
Should Be Equal ${results[0].uko_node} project://app/db.py
|
|
${count}= Get Length ${results}
|
|
Should Be True ${count} >= 1
|
|
|
|
Semantic Search Filters Low Similarity
|
|
[Documentation] Verify semantic search filters low-similarity results
|
|
[Tags] semantic_search integration
|
|
${strategy}= Create Semantic Search Strategy
|
|
${fragments}= Create Test Fragments
|
|
... project://app/db.py database handler 0.9 20 3
|
|
... project://app/io.py file io module 0.8 15 3
|
|
${budget}= Create Context Budget 1000 0
|
|
${results}= Search With Query ${strategy} quantum computing ${fragments} ${budget}
|
|
Length Should Be ${results} 0
|
|
|
|
Relevance Scoring Ranks By Score
|
|
[Documentation] Verify relevance scoring ranks by relevance score
|
|
[Tags] relevance_scoring integration
|
|
${strategy}= Create Relevance Scoring Strategy
|
|
${fragments}= Create Test Fragments
|
|
... project://app/a.py alpha 0.3 10 3
|
|
... project://app/b.py beta 0.9 10 3
|
|
... project://app/c.py gamma 0.6 10 3
|
|
${budget}= Create Context Budget 1000 0
|
|
${results}= Assemble With Strategy ${strategy} ${fragments} ${budget}
|
|
Should Be Equal ${results[0].uko_node} project://app/b.py
|
|
Should Be Equal ${results[1].uko_node} project://app/c.py
|
|
Should Be Equal ${results[2].uko_node} project://app/a.py
|
|
|
|
Relevance Scoring Respects Budget
|
|
[Documentation] Verify relevance scoring respects token budget
|
|
[Tags] relevance_scoring integration
|
|
${strategy}= Create Relevance Scoring Strategy
|
|
${fragments}= Create Test Fragments
|
|
... project://app/a.py alpha 0.9 100 3
|
|
... project://app/b.py beta 0.8 100 3
|
|
... project://app/c.py gamma 0.7 100 3
|
|
${budget}= Create Context Budget 250 0
|
|
${results}= Assemble With Strategy ${strategy} ${fragments} ${budget}
|
|
Length Should Be ${results} 2
|
|
|
|
Adaptive Selector Chooses Best Strategy
|
|
[Documentation] Verify adaptive selector chooses best strategy for query
|
|
[Tags] adaptive_selector integration
|
|
${selector}= Create Adaptive Selector
|
|
${fragments}= Create Test Fragments
|
|
... project://app/db.py Database connection pool manager 0.5 20 3
|
|
... project://app/io.py File input output handler 0.8 15 3
|
|
... project://app/sql.py SQL database query executor 0.6 25 3
|
|
${budget}= Create Context Budget 1000 0
|
|
${strategy_name}= Select Strategy For Query ${selector} database connection
|
|
Should Be Equal ${strategy_name} semantic-embedding
|
|
|
|
Adaptive Selector Falls Back To Relevance
|
|
[Documentation] Verify adaptive selector falls back to relevance without query
|
|
[Tags] adaptive_selector integration
|
|
${selector}= Create Adaptive Selector
|
|
${fragments}= Create Test Fragments
|
|
... project://app/a.py alpha 0.3 10 3
|
|
... project://app/b.py beta 0.9 10 3
|
|
${budget}= Create Context Budget 1000 0
|
|
${strategy_name}= Select Strategy Without Query ${selector}
|
|
Should Be Equal ${strategy_name} relevance-scoring
|
|
|
|
Context Fusion Combines Results
|
|
[Documentation] Verify context fusion combines results from multiple strategies
|
|
[Tags] context_fusion integration
|
|
${fusion}= Create Context Fusion Strategy semantic-embedding relevance-scoring
|
|
${fragments}= Create Test Fragments
|
|
... project://app/db.py Database connection pool manager 0.5 20 3
|
|
... project://app/io.py File input output handler 0.8 15 3
|
|
... project://app/sql.py SQL database query executor 0.6 25 3
|
|
${budget}= Create Context Budget 1000 0
|
|
${results}= Fuse With Query ${fusion} database ${fragments} ${budget}
|
|
${count}= Get Length ${results}
|
|
Should Be True ${count} >= 2
|
|
|
|
Context Fusion Deduplicates
|
|
[Documentation] Verify context fusion deduplicates results
|
|
[Tags] context_fusion integration
|
|
${fusion}= Create Context Fusion Strategy semantic-embedding relevance-scoring
|
|
${fragments}= Create Test Fragments
|
|
... project://app/a.py database 0.9 100 3
|
|
... project://app/b.py database 0.8 100 3
|
|
${budget}= Create Context Budget 1000 0
|
|
${results}= Fuse With Query ${fusion} database ${fragments} ${budget}
|
|
${uko_nodes}= Get Uko Nodes ${results}
|
|
${unique_nodes}= Get Length ${uko_nodes}
|
|
${total_nodes}= Get Length ${uko_nodes}
|
|
Should Be Equal ${unique_nodes} ${total_nodes}
|
|
|
|
YAML Configuration Loads Semantic Search
|
|
[Documentation] Verify YAML configuration loads semantic search strategy
|
|
[Tags] yaml_config integration
|
|
${config}= Create Dictionary strategy=semantic-embedding min_similarity=0.05
|
|
${strategy}= Load Strategy From YAML ${config}
|
|
Should Be Equal ${strategy.name} semantic-embedding
|
|
|
|
YAML Configuration Loads Relevance Scoring
|
|
[Documentation] Verify YAML configuration loads relevance scoring strategy
|
|
[Tags] yaml_config integration
|
|
${config}= Create Dictionary strategy=relevance-scoring
|
|
${strategy}= Load Strategy From YAML ${config}
|
|
Should Be Equal ${strategy.name} relevance-scoring
|
|
|
|
YAML Configuration Loads Adaptive Selector
|
|
[Documentation] Verify YAML configuration loads adaptive selector
|
|
[Tags] yaml_config integration
|
|
${config}= Create Dictionary strategy=adaptive-selector fallback=relevance-scoring
|
|
${strategy}= Load Strategy From YAML ${config}
|
|
Should Be Equal ${strategy.name} adaptive-selector
|
|
|
|
YAML Configuration Loads Context Fusion
|
|
[Documentation] Verify YAML configuration loads context fusion
|
|
[Tags] yaml_config integration
|
|
${strategies}= Create List semantic-embedding relevance-scoring
|
|
${config}= Create Dictionary strategy=context-fusion strategies=${strategies}
|
|
${strategy}= Load Strategy From YAML ${config}
|
|
Should Be Equal ${strategy.name} context-fusion
|
|
|
|
ContextAssembler Integrates Advanced Strategies
|
|
[Documentation] Verify ContextAssembler integrates advanced strategies
|
|
[Tags] integration assembler
|
|
${assembler}= Create Context Assembler With Advanced Strategies
|
|
${fragments}= Create Test Fragments
|
|
... project://app/db.py Database connection pool manager 0.5 20 3
|
|
... project://app/io.py File input output handler 0.8 15 3
|
|
... project://app/sql.py SQL database query executor 0.6 25 3
|
|
${budget}= Create Context Budget 1000 0
|
|
${results}= Assemble Context With Query ${assembler} database ${fragments} ${budget}
|
|
${count}= Get Length ${results}
|
|
Should Be True ${count} > 0
|
|
|
|
ContextAssembler Respects Strategy Priority
|
|
[Documentation] Verify ContextAssembler respects strategy priority
|
|
[Tags] integration assembler
|
|
${assembler}= Create Context Assembler With Multiple Strategies
|
|
${fragments}= Create Test Fragments
|
|
... project://app/a.py alpha module 0.5 10 3
|
|
... project://app/b.py beta module 0.9 10 3
|
|
${budget}= Create Context Budget 1000 0
|
|
${results}= Assemble Context With Query ${assembler} module ${fragments} ${budget}
|
|
${count}= Get Length ${results}
|
|
Should Be True ${count} > 0
|
|
|
|
Semantic Search Handles Empty Query
|
|
[Documentation] Verify semantic search handles empty query gracefully
|
|
[Tags] error_handling integration
|
|
${strategy}= Create Semantic Search Strategy
|
|
${fragments}= Create Test Fragments
|
|
... project://app/a.py alpha 0.5 10 3
|
|
${budget}= Create Context Budget 1000 0
|
|
${results}= Search With Query ${strategy} ${EMPTY} ${fragments} ${budget}
|
|
${count}= Get Length ${results}
|
|
Should Be True ${count} >= 0
|
|
|
|
Adaptive Selector Handles Invalid Request
|
|
[Documentation] Verify adaptive selector handles invalid request
|
|
[Tags] error_handling integration
|
|
${selector}= Create Adaptive Selector
|
|
${strategy_name}= Select Strategy With Invalid Request ${selector}
|
|
Should Not Be Empty ${strategy_name}
|
|
|
|
*** Keywords ***
|
|
Create Semantic Search Strategy
|
|
[Documentation] Create a semantic search strategy with FakeEmbeddings
|
|
${strategy}= Create Semantic Search Strategy Impl
|
|
RETURN ${strategy}
|
|
|
|
Create Relevance Scoring Strategy
|
|
[Documentation] Create a relevance scoring strategy
|
|
${strategy}= Create Relevance Scoring Strategy Impl
|
|
RETURN ${strategy}
|
|
|
|
Create Adaptive Selector
|
|
[Documentation] Create an adaptive context strategy selector
|
|
${selector}= Create Adaptive Selector Impl
|
|
RETURN ${selector}
|
|
|
|
Create Context Fusion Strategy
|
|
[Documentation] Create a context fusion strategy
|
|
[Arguments] @{strategies}
|
|
${fusion}= Create Context Fusion Strategy Impl ${strategies}
|
|
RETURN ${fusion}
|
|
|
|
Create Test Fragments
|
|
[Documentation] Create test context fragments
|
|
[Arguments] @{args}
|
|
${fragments}= Create Test Fragments Impl ${args}
|
|
RETURN ${fragments}
|
|
|
|
Create Context Budget
|
|
[Documentation] Create a context budget
|
|
[Arguments] ${max_tokens} ${reserved_tokens}
|
|
${budget}= Create Context Budget Impl ${max_tokens} ${reserved_tokens}
|
|
RETURN ${budget}
|
|
|
|
Search With Query
|
|
[Documentation] Search with a query
|
|
[Arguments] ${strategy} ${query} ${fragments} ${budget}
|
|
${results}= Search With Query Impl ${strategy} ${query} ${fragments} ${budget}
|
|
RETURN ${results}
|
|
|
|
Assemble With Strategy
|
|
[Documentation] Assemble with a strategy
|
|
[Arguments] ${strategy} ${fragments} ${budget}
|
|
${results}= Assemble With Strategy Impl ${strategy} ${fragments} ${budget}
|
|
RETURN ${results}
|
|
|
|
Select Strategy For Query
|
|
[Documentation] Select strategy for a query
|
|
[Arguments] ${selector} ${query}
|
|
${strategy_name}= Select Strategy For Query Impl ${selector} ${query}
|
|
RETURN ${strategy_name}
|
|
|
|
Select Strategy Without Query
|
|
[Documentation] Select strategy without query
|
|
[Arguments] ${selector}
|
|
${strategy_name}= Select Strategy Without Query Impl ${selector}
|
|
RETURN ${strategy_name}
|
|
|
|
Select Strategy With Invalid Request
|
|
[Documentation] Select strategy with invalid request
|
|
[Arguments] ${selector}
|
|
${strategy_name}= Select Strategy With Invalid Request Impl ${selector}
|
|
RETURN ${strategy_name}
|
|
|
|
Fuse With Query
|
|
[Documentation] Fuse strategies with a query
|
|
[Arguments] ${fusion} ${query} ${fragments} ${budget}
|
|
${results}= Fuse With Query Impl ${fusion} ${query} ${fragments} ${budget}
|
|
RETURN ${results}
|
|
|
|
Load Strategy From YAML
|
|
[Documentation] Load strategy from YAML configuration
|
|
[Arguments] ${config}
|
|
${strategy}= Load Strategy From YAML Impl ${config}
|
|
RETURN ${strategy}
|
|
|
|
Create Context Assembler With Advanced Strategies
|
|
[Documentation] Create a ContextAssembler with advanced strategies
|
|
${assembler}= Create Context Assembler With Advanced Strategies Impl
|
|
RETURN ${assembler}
|
|
|
|
Create Context Assembler With Multiple Strategies
|
|
[Documentation] Create a ContextAssembler with multiple strategies
|
|
${assembler}= Create Context Assembler With Multiple Strategies Impl
|
|
RETURN ${assembler}
|
|
|
|
Assemble Context With Query
|
|
[Documentation] Assemble context with a query
|
|
[Arguments] ${assembler} ${query} ${fragments} ${budget}
|
|
${results}= Assemble Context With Query Impl ${assembler} ${query} ${fragments} ${budget}
|
|
RETURN ${results}
|
|
|
|
Get Uko Nodes
|
|
[Documentation] Extract uko_node values from fragments
|
|
[Arguments] ${fragments}
|
|
${nodes}= Get Uko Nodes Impl ${fragments}
|
|
RETURN ${nodes}
|