freemo
487e16a9f0
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 13s
CI / build (pull_request) Successful in 16s
CI / quality (pull_request) Successful in 17s
CI / security (pull_request) Successful in 32s
CI / typecheck (pull_request) Successful in 33s
CI / unit_tests (pull_request) Successful in 2m7s
CI / docker (pull_request) Successful in 37s
CI / integration_tests (pull_request) Successful in 2m56s
CI / coverage (pull_request) Successful in 4m26s
CI / benchmark-regression (pull_request) Successful in 28m12s
CI / lint (push) Successful in 12s
CI / build (push) Successful in 15s
CI / quality (push) Successful in 16s
CI / security (push) Successful in 31s
CI / typecheck (push) Successful in 34s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 2m4s
CI / docker (push) Successful in 38s
CI / integration_tests (push) Successful in 3m7s
CI / coverage (push) Successful in 5m4s
CI / benchmark-publish (push) Has been cancelled
Implement the first three built-in context strategies for the ACMS v1 context assembly pipeline: 1. SimpleKeywordStrategy (quality 0.3) - Keyword matching on fragment content with word-density fallback. Universal fallback strategy. 2. SemanticEmbeddingStrategy (quality 0.6) - Jaccard word-overlap similarity scoring between query and fragment content. 3. BreadthDepthNavigatorStrategy (quality 0.85) - UKO node hierarchy navigation prioritising fragments near focus nodes with higher detail depths. Primary strategy for code projects. All strategies implement the v1 ContextStrategy Protocol from acms_service.py and can be registered with ACMSPipeline via register_strategy(). Includes: - 28 Behave BDD scenarios covering ranking, budget, capabilities, can_handle confidence, explain, empty input, and pipeline registration - 9 Robot Framework integration tests - ASV benchmarks at 10/100/1000 fragment scales for all 3 strategies - Vulture whitelist entries for public API symbols - 100% coverage on context_strategies.py ISSUES CLOSED: #541
82 lines
3.8 KiB
Plaintext
82 lines
3.8 KiB
Plaintext
*** Settings ***
|
|
Documentation Integration tests for built-in context strategies batch 1
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${HELPER} ${CURDIR}/helper_context_strategies.py
|
|
|
|
*** Test Cases ***
|
|
SimpleKeyword Ranks By Keyword Match
|
|
[Documentation] SimpleKeywordStrategy ranks keyword-matching fragments first
|
|
${result}= Run Process ${PYTHON} ${HELPER} simple-keyword-rank cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} context-strategies-ok: simple-keyword-rank
|
|
|
|
SimpleKeyword Respects Budget
|
|
[Documentation] SimpleKeywordStrategy respects token budget
|
|
${result}= Run Process ${PYTHON} ${HELPER} simple-keyword-budget cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} context-strategies-ok: simple-keyword-budget
|
|
|
|
SemanticEmbedding Ranks By Similarity
|
|
[Documentation] SemanticEmbeddingStrategy ranks by word similarity
|
|
${result}= Run Process ${PYTHON} ${HELPER} semantic-embedding-rank cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} context-strategies-ok: semantic-embedding-rank
|
|
|
|
SemanticEmbedding Filters Unrelated
|
|
[Documentation] SemanticEmbeddingStrategy filters below similarity threshold
|
|
${result}= Run Process ${PYTHON} ${HELPER} semantic-embedding-filter cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} context-strategies-ok: semantic-embedding-filter
|
|
|
|
BreadthDepth Prioritises Near Focus
|
|
[Documentation] BreadthDepthNavigatorStrategy prioritises fragments near focus
|
|
${result}= Run Process ${PYTHON} ${HELPER} breadth-depth-rank cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} context-strategies-ok: breadth-depth-rank
|
|
|
|
BreadthDepth Respects Budget
|
|
[Documentation] BreadthDepthNavigatorStrategy respects token budget
|
|
${result}= Run Process ${PYTHON} ${HELPER} breadth-depth-budget cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} context-strategies-ok: breadth-depth-budget
|
|
|
|
Pipeline Registration
|
|
[Documentation] Register all batch 1 strategies with pipeline
|
|
${result}= Run Process ${PYTHON} ${HELPER} pipeline-register cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} context-strategies-ok: pipeline-register
|
|
|
|
Can Handle Confidence Values
|
|
[Documentation] Verify can_handle confidence values for all strategies
|
|
${result}= Run Process ${PYTHON} ${HELPER} can-handle cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} context-strategies-ok: can-handle
|
|
|
|
Strategy Capabilities
|
|
[Documentation] Verify capability flags and names for all strategies
|
|
${result}= Run Process ${PYTHON} ${HELPER} capabilities cwd=${WORKSPACE}
|
|
Log ${result.stdout}
|
|
Log ${result.stderr}
|
|
Should Be Equal As Integers ${result.rc} 0
|
|
Should Contain ${result.stdout} context-strategies-ok: capabilities
|