freemo
fe7381c45b
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 14s
CI / build (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 27s
CI / security (pull_request) Successful in 32s
CI / typecheck (pull_request) Successful in 36s
CI / unit_tests (pull_request) Successful in 3m27s
CI / integration_tests (pull_request) Successful in 3m36s
CI / docker (pull_request) Successful in 42s
CI / coverage (pull_request) Successful in 4m50s
CI / lint (push) Successful in 13s
CI / build (push) Successful in 14s
CI / quality (push) Successful in 16s
CI / security (push) Successful in 32s
CI / typecheck (push) Successful in 36s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 2m1s
CI / docker (push) Successful in 51s
CI / integration_tests (push) Successful in 3m7s
CI / coverage (push) Successful in 4m23s
CI / benchmark-publish (push) Has been cancelled
CI / benchmark-regression (pull_request) Successful in 29m6s
Add production-grade Phase 2 (Fragment Fusion) components for the ACMS context assembly pipeline, replacing the no-op defaults: - ContentHashDeduplicator: Groups fragments by UKO node URI, hashes content to detect duplicates, retains highest relevance_score. - MaxDepthResolver: Resolves depth conflicts by keeping the highest detail depth per UKO node, with relevance tiebreaking. - WeightedCompositeScorer: Computes composite score from configurable weighted factors (relevance=0.4, hierarchy=0.3, quality=0.2, recency=0.1). Stores component breakdown in metadata. - GreedyKnapsackPacker: Greedy knapsack selection with depth fallback (tries depths [9,4,2,0] for oversized fragments) and minimum fragment token threshold (10). Also adds: - ScoredFragment frozen Pydantic model (spec §42825) with composite_score, score_components, and fragment reference - score_detailed() method on WeightedCompositeScorer returning ScoredFragment objects for callers needing full breakdowns - All components implement v1 Protocol signatures from acms_service.py and can be DI-injected into ACMSPipeline constructor Testing: - 31 Behave BDD scenarios in acms_pipeline_phase2.feature covering deduplication, depth resolution, scoring, packing, depth fallback, budget constraints, pipeline integration, and ScoredFragment model - 6 Robot Framework integration smoke tests - ASV benchmark suites for all 4 components and ScoredFragment Quality gates: lint, typecheck (0 errors), unit_tests (8555 scenarios), coverage (97.0%), dead_code — all passing. ISSUES CLOSED: #540
324 lines
15 KiB
Gherkin
324 lines
15 KiB
Gherkin
@phase2 @acms @acms_pipeline_phase2
|
|
Feature: ACMS Pipeline Phase 2 — Fragment Fusion Components
|
|
As a CleverAgents developer
|
|
I want production-grade Phase 2 pipeline components
|
|
So that context assembly deduplicates, resolves depths, scores, and packs fragments
|
|
|
|
# ===========================================================================
|
|
# ScoredFragment — Domain Model
|
|
# ===========================================================================
|
|
|
|
@scored_fragment
|
|
Scenario: Create a ScoredFragment with component breakdown
|
|
Given a context fragment with uko_node "project://app/main.py" and content "hello" and token_count 10 for phase2
|
|
When I create a ScoredFragment with composite_score 0.85 and components:
|
|
| component | value |
|
|
| relevance | 0.9 |
|
|
| hierarchy | 0.8 |
|
|
| quality | 0.7 |
|
|
| recency | 1.0 |
|
|
Then the ScoredFragment composite_score should be 0.85
|
|
And the ScoredFragment should have component "relevance" with value 0.9
|
|
And the ScoredFragment should have component "hierarchy" with value 0.8
|
|
And the ScoredFragment should have component "quality" with value 0.7
|
|
And the ScoredFragment should have component "recency" with value 1.0
|
|
And the ScoredFragment should reference the original fragment
|
|
|
|
@scored_fragment @validation
|
|
Scenario: ScoredFragment rejects composite score above 1.0
|
|
Given a context fragment with uko_node "project://app/main.py" and content "hello" and token_count 10 for phase2
|
|
When I create a ScoredFragment with invalid composite_score 1.5
|
|
Then a phase2 validation error should be raised
|
|
|
|
@scored_fragment @validation
|
|
Scenario: ScoredFragment rejects negative composite score
|
|
Given a context fragment with uko_node "project://app/main.py" and content "hello" and token_count 10 for phase2
|
|
When I create a ScoredFragment with invalid composite_score -0.1
|
|
Then a phase2 validation error should be raised
|
|
|
|
# ===========================================================================
|
|
# ContentHashDeduplicator
|
|
# ===========================================================================
|
|
|
|
@deduplicator
|
|
Scenario: Deduplicate fragments with identical content for same UKO node
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | hello world | 0.5 | 10 | 3 |
|
|
| project://app/main.py | hello world | 0.9 | 10 | 3 |
|
|
When I deduplicate the fragments
|
|
Then 1 fragment should remain after deduplication
|
|
And the surviving fragment should have relevance score 0.9
|
|
|
|
@deduplicator
|
|
Scenario: Deduplicate keeps distinct content for same UKO node
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | hello world | 0.5 | 10 | 3 |
|
|
| project://app/main.py | goodbye world | 0.9 | 12 | 3 |
|
|
When I deduplicate the fragments
|
|
Then 2 fragments should remain after deduplication
|
|
|
|
@deduplicator
|
|
Scenario: Deduplicate keeps fragments from different UKO nodes
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | hello world | 0.5 | 10 | 3 |
|
|
| project://app/io.py | hello world | 0.9 | 10 | 3 |
|
|
When I deduplicate the fragments
|
|
Then 2 fragments should remain after deduplication
|
|
|
|
@deduplicator
|
|
Scenario: Deduplicate with empty fragment list
|
|
Given an empty phase2 fragment list
|
|
When I deduplicate the fragments
|
|
Then 0 fragments should remain after deduplication
|
|
|
|
@deduplicator
|
|
Scenario: Deduplicate three copies retains highest scored
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | abc | 0.3 | 5 | 1 |
|
|
| project://app/main.py | abc | 0.7 | 5 | 1 |
|
|
| project://app/main.py | abc | 0.5 | 5 | 1 |
|
|
When I deduplicate the fragments
|
|
Then 1 fragment should remain after deduplication
|
|
And the surviving fragment should have relevance score 0.7
|
|
|
|
# ===========================================================================
|
|
# MaxDepthResolver
|
|
# ===========================================================================
|
|
|
|
@depth_resolver
|
|
Scenario: Resolve depth conflicts to maximum depth per node
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | brief summary | 0.8 | 10 | 2 |
|
|
| project://app/main.py | full source | 0.8 | 100 | 9 |
|
|
When I resolve depth conflicts
|
|
Then 1 fragment should remain after depth resolution
|
|
And the surviving fragment should have detail depth 9
|
|
|
|
@depth_resolver
|
|
Scenario: Resolve preserves fragments for distinct nodes
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | alpha | 0.8 | 10 | 2 |
|
|
| project://app/io.py | beta | 0.7 | 20 | 5 |
|
|
When I resolve depth conflicts
|
|
Then 2 fragments should remain after depth resolution
|
|
|
|
@depth_resolver
|
|
Scenario: Resolve same depth keeps higher relevance
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | alpha | 0.3 | 10 | 5 |
|
|
| project://app/main.py | beta | 0.9 | 20 | 5 |
|
|
When I resolve depth conflicts
|
|
Then 1 fragment should remain after depth resolution
|
|
And the surviving fragment should have relevance score 0.9
|
|
|
|
@depth_resolver
|
|
Scenario: Resolve with empty fragment list
|
|
Given an empty phase2 fragment list
|
|
When I resolve depth conflicts
|
|
Then 0 fragments should remain after depth resolution
|
|
|
|
@depth_resolver
|
|
Scenario: Resolve preserves original fragment order
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | aaa | 0.5 | 10 | 3 |
|
|
| project://app/b.py | bbb | 0.6 | 10 | 3 |
|
|
| project://app/c.py | ccc | 0.7 | 10 | 3 |
|
|
When I resolve depth conflicts
|
|
Then 3 fragments should remain after depth resolution
|
|
And the resolved fragment order should be "aaa", "bbb", "ccc"
|
|
|
|
# ===========================================================================
|
|
# WeightedCompositeScorer
|
|
# ===========================================================================
|
|
|
|
@scorer
|
|
Scenario: Score fragments with default weights
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | hello | 0.8 | 10 | 3 |
|
|
When I score the fragments with default weights
|
|
Then the scored fragment should have a composite relevance score
|
|
And the scored fragment metadata should contain original relevance "0.8"
|
|
|
|
@scorer
|
|
Scenario: Score with custom weights
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | hello | 1.0 | 10 | 3 |
|
|
And scorer weights: relevance=1.0, hierarchy=0.0, quality=0.0, recency=0.0
|
|
When I score the fragments with custom weights
|
|
Then the scored fragment relevance score should be 1.0
|
|
|
|
@scorer
|
|
Scenario: Score with metadata-provided hierarchy weight
|
|
Given a phase2 fragment with metadata:
|
|
| uko_node | content | score | tokens | depth | hierarchy_weight | strategy_quality | recency_bonus |
|
|
| project://app/main.py | hello | 0.8 | 10 | 3 | 0.9 | 0.7 | 1.0 |
|
|
When I score the fragments with default weights
|
|
Then the scored fragment metadata should contain component "hierarchy" near 0.9
|
|
|
|
@scorer
|
|
Scenario: Score empty fragment list
|
|
Given an empty phase2 fragment list
|
|
When I score the fragments with default weights
|
|
Then 0 scored fragments should be returned
|
|
|
|
@scorer
|
|
Scenario: Detailed scoring returns ScoredFragment objects
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | hello | 0.8 | 10 | 3 |
|
|
| project://app/io.py | world | 0.6 | 15 | 5 |
|
|
When I score the fragments using score_detailed
|
|
Then 2 ScoredFragment objects should be returned
|
|
And each ScoredFragment should have a composite_score between 0.0 and 1.0
|
|
And each ScoredFragment should have score_components
|
|
|
|
@scorer
|
|
Scenario: Score composite is clamped to 0.0-1.0
|
|
Given a phase2 fragment with metadata:
|
|
| uko_node | content | score | tokens | depth | hierarchy_weight | strategy_quality | recency_bonus |
|
|
| project://app/main.py | hello | 1.0 | 10 | 3 | 1.0 | 1.0 | 1.0 |
|
|
When I score the fragments with default weights
|
|
Then the scored fragment relevance score should be at most 1.0
|
|
|
|
# ===========================================================================
|
|
# GreedyKnapsackPacker
|
|
# ===========================================================================
|
|
|
|
@packer
|
|
Scenario: Pack fragments within budget
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | alpha | 0.9 | 100 | 5 |
|
|
| project://app/io.py | beta | 0.7 | 100 | 3 |
|
|
| project://app/util.py | gamma | 0.5 | 100 | 2 |
|
|
And a phase2 budget with max_tokens 250 and reserved_tokens 0
|
|
When I pack the fragments into budget
|
|
Then 2 fragments should be packed
|
|
And the packed total tokens should be at most 250
|
|
|
|
@packer
|
|
Scenario: Pack all fragments when budget is sufficient
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | alpha | 0.9 | 50 | 5 |
|
|
| project://app/io.py | beta | 0.7 | 50 | 3 |
|
|
And a phase2 budget with max_tokens 200 and reserved_tokens 0
|
|
When I pack the fragments into budget
|
|
Then 2 fragments should be packed
|
|
|
|
@packer
|
|
Scenario: Pack empty fragment list
|
|
Given an empty phase2 fragment list
|
|
And a phase2 budget with max_tokens 1000 and reserved_tokens 0
|
|
When I pack the fragments into budget
|
|
Then 0 fragments should be packed
|
|
|
|
@packer
|
|
Scenario: Pack excludes fragments below minimum token threshold
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | a | 0.9 | 5 | 3 |
|
|
| project://app/io.py | beta | 0.7 | 50 | 3 |
|
|
And a phase2 budget with max_tokens 1000 and reserved_tokens 0
|
|
When I pack the fragments into budget
|
|
Then 1 fragment should be packed
|
|
And the packed fragment uko_node should be "project://app/io.py"
|
|
|
|
@packer
|
|
Scenario: Pack with depth fallback uses lower-depth alternative
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | full source code | 0.9 | 500 | 9 |
|
|
| project://app/main.py | brief summary | 0.5 | 50 | 2 |
|
|
| project://app/io.py | io module | 0.7 | 100 | 5 |
|
|
And a phase2 budget with max_tokens 200 and reserved_tokens 0
|
|
When I pack the fragments into budget
|
|
Then the packed fragments should include uko_node "project://app/main.py"
|
|
And the packed fragment for "project://app/main.py" should have depth 2
|
|
|
|
@packer
|
|
Scenario: Pack respects reserved tokens
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | alpha | 0.9 | 100 | 5 |
|
|
| project://app/io.py | beta | 0.7 | 100 | 3 |
|
|
And a phase2 budget with max_tokens 250 and reserved_tokens 100
|
|
When I pack the fragments into budget
|
|
Then 1 fragment should be packed
|
|
And the packed total tokens should be at most 150
|
|
|
|
@packer
|
|
Scenario: Pack prefers higher scored fragments
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | alpha | 0.3 | 100 | 5 |
|
|
| project://app/io.py | beta | 0.9 | 100 | 3 |
|
|
And a phase2 budget with max_tokens 150 and reserved_tokens 0
|
|
When I pack the fragments into budget
|
|
Then 1 fragment should be packed
|
|
And the packed fragment uko_node should be "project://app/io.py"
|
|
|
|
@packer
|
|
Scenario: Single fragment over entire budget is excluded
|
|
Given the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | huge content | 0.9 | 10000 | 9 |
|
|
And a phase2 budget with max_tokens 100 and reserved_tokens 0
|
|
When I pack the fragments into budget
|
|
Then 0 fragments should be packed
|
|
|
|
# ===========================================================================
|
|
# Pipeline Integration — DI injection of Phase 2 components
|
|
# ===========================================================================
|
|
|
|
@pipeline_integration
|
|
Scenario: Inject ContentHashDeduplicator into pipeline
|
|
Given the ACMS pipeline with a ContentHashDeduplicator
|
|
And the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | hello world | 0.5 | 10 | 3 |
|
|
| project://app/main.py | hello world | 0.9 | 10 | 3 |
|
|
When I assemble context through the pipeline
|
|
Then the pipeline output should contain 1 fragment
|
|
|
|
@pipeline_integration
|
|
Scenario: Inject MaxDepthResolver into pipeline
|
|
Given the ACMS pipeline with a MaxDepthResolver
|
|
And the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | brief | 0.8 | 10 | 2 |
|
|
| project://app/main.py | detailed | 0.8 | 50 | 9 |
|
|
When I assemble context through the pipeline
|
|
Then the pipeline output should contain 1 fragment
|
|
|
|
@pipeline_integration
|
|
Scenario: Inject WeightedCompositeScorer into pipeline
|
|
Given the ACMS pipeline with a WeightedCompositeScorer
|
|
And the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | hello | 0.8 | 10 | 3 |
|
|
When I assemble context through the pipeline
|
|
Then the pipeline output fragments should have updated relevance scores
|
|
|
|
@pipeline_integration
|
|
Scenario: Inject GreedyKnapsackPacker into pipeline
|
|
Given the ACMS pipeline with a GreedyKnapsackPacker
|
|
And the following phase2 fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/main.py | alpha | 0.9 | 100 | 5 |
|
|
| project://app/io.py | beta | 0.7 | 100 | 3 |
|
|
| project://app/util.py | gamma | 0.5 | 100 | 2 |
|
|
And a phase2 budget with max_tokens 250 and reserved_tokens 0
|
|
When I assemble context through the pipeline with budget
|
|
Then the pipeline output total tokens should be at most 250
|