@phase2 @acms @context_strategies @relevance_scoring Feature: RelevanceScoringStrategy for Context File Selection As a CleverAgents developer I want relevance scoring for context file selection So that context files are ranked by semantic relevance, recency, and importance # =========================================================================== # RelevanceScoringStrategy Basic Functionality # =========================================================================== @relevance_scoring Scenario: RelevanceScoring ranks by semantic similarity Given a RelevanceScoringStrategy with query "database connection" And the following strategy fragments: | uko_node | content | score | tokens | depth | | project://app/db.py | Database connection pool manager | 0.8 | 20 | 5 | | project://app/io.py | File input output handler | 0.5 | 15 | 3 | | project://app/sql.py | SQL database query executor | 0.7 | 25 | 4 | And a strategy budget with max_tokens 1000 and reserved_tokens 0 When I assemble with the RelevanceScoringStrategy Then the first result fragment should have uko_node "project://app/db.py" @relevance_scoring Scenario: RelevanceScoring factors in recency Given a RelevanceScoringStrategy with query "async" And the following strategy fragments: | uko_node | content | score | tokens | depth | | project://app/old.py | async old code | 0.3 | 10 | 2 | | project://app/new.py | async new code | 0.9 | 10 | 2 | And a strategy budget with max_tokens 1000 and reserved_tokens 0 When I assemble with the RelevanceScoringStrategy Then the first result fragment should have uko_node "project://app/new.py" @relevance_scoring Scenario: RelevanceScoring factors in importance (depth) Given a RelevanceScoringStrategy with query "core" And the following strategy fragments: | uko_node | content | score | tokens | depth | | project://app/a.py | core module | 0.5 | 10 | 9 | | project://app/b.py | core module | 0.5 | 10 | 1 | And a strategy budget with max_tokens 1000 and reserved_tokens 0 When I assemble with the RelevanceScoringStrategy Then the first result fragment should have uko_node "project://app/a.py" @relevance_scoring Scenario: RelevanceScoring without query falls back to metadata Given a RelevanceScoringStrategy without query And the following strategy fragments: | uko_node | content | score | tokens | depth | | project://app/a.py | alpha | 0.3 | 10 | 3 | | project://app/b.py | beta | 0.9 | 10 | 3 | And a strategy budget with max_tokens 1000 and reserved_tokens 0 When I assemble with the RelevanceScoringStrategy Then the first result fragment should have uko_node "project://app/b.py" @relevance_scoring Scenario: RelevanceScoring can_handle returns 0.7 with query Given a RelevanceScoringStrategy without query When I check can_handle on RelevanceScoringStrategy with query "test" Then the strategy confidence should be 0.7 @relevance_scoring Scenario: RelevanceScoring can_handle returns 0.2 without query Given a RelevanceScoringStrategy without query When I check can_handle on RelevanceScoringStrategy without query Then the strategy confidence should be 0.2 @relevance_scoring Scenario: RelevanceScoring reports capabilities Given a RelevanceScoringStrategy without query Then the RelevanceScoringStrategy should support semantic search And the RelevanceScoringStrategy name should be "relevance-scoring" @relevance_scoring Scenario: RelevanceScoring respects budget Given a RelevanceScoringStrategy with query "hello" And the following strategy fragments: | uko_node | content | score | tokens | depth | | project://app/a.py | hello world | 0.9 | 100 | 5 | | project://app/b.py | hello there | 0.7 | 100 | 4 | | project://app/c.py | hello again | 0.5 | 100 | 3 | And a strategy budget with max_tokens 250 and reserved_tokens 0 When I assemble with the RelevanceScoringStrategy Then 2 fragments should be returned by strategy @relevance_scoring Scenario: RelevanceScoring returns empty for empty input Given a RelevanceScoringStrategy with query "test" And an empty strategy fragment list And a strategy budget with max_tokens 1000 and reserved_tokens 0 When I assemble with the RelevanceScoringStrategy Then 0 fragments should be returned by strategy @relevance_scoring Scenario: RelevanceScoring handles fragment with empty content Given a RelevanceScoringStrategy with query "test" And the following strategy fragments: | uko_node | content | score | tokens | depth | | project://app/a.py | | 0.5 | 10 | 3 | And a strategy budget with max_tokens 1000 and reserved_tokens 0 When I assemble with the RelevanceScoringStrategy Then 1 fragments should be returned by strategy @relevance_scoring Scenario: RelevanceScoring explain returns description Given a RelevanceScoringStrategy without query Then the RelevanceScoringStrategy explain should contain "relevance" # =========================================================================== # RelevanceScoringStrategy Weight Configuration # =========================================================================== @relevance_scoring Scenario: RelevanceScoring with custom similarity weight Given a RelevanceScoringStrategy with similarity_weight 0.8 and recency_weight 0.1 and importance_weight 0.1 And the following strategy fragments: | uko_node | content | score | tokens | depth | | project://app/a.py | database connection | 0.9 | 10 | 1 | | project://app/b.py | database connection | 0.5 | 10 | 9 | And a strategy budget with max_tokens 1000 and reserved_tokens 0 When I assemble with the RelevanceScoringStrategy Then the first result fragment should have uko_node "project://app/a.py" @relevance_scoring Scenario: RelevanceScoring with custom importance weight Given a RelevanceScoringStrategy with similarity_weight 0.1 and recency_weight 0.1 and importance_weight 0.8 And the following strategy fragments: | uko_node | content | score | tokens | depth | | project://app/a.py | test | 0.3 | 10 | 9 | | project://app/b.py | test | 0.9 | 10 | 1 | And a strategy budget with max_tokens 1000 and reserved_tokens 0 When I assemble with the RelevanceScoringStrategy Then the first result fragment should have uko_node "project://app/a.py" # =========================================================================== # Pipeline Registration # =========================================================================== @registration Scenario: Register RelevanceScoringStrategy with pipeline Given an ACMS pipeline for strategy tests When I register RelevanceScoringStrategy with the pipeline Then the pipeline should have strategy "relevance-scoring"