HAL9000
a757633e27
CI / lint (pull_request) Successful in 44s
CI / build (pull_request) Successful in 43s
CI / helm (pull_request) Successful in 1m10s
CI / quality (pull_request) Successful in 1m15s
CI / typecheck (pull_request) Successful in 1m17s
CI / security (pull_request) Successful in 1m18s
CI / push-validation (pull_request) Successful in 25s
CI / unit_tests (pull_request) Successful in 6m34s
CI / docker (pull_request) Successful in 1m45s
CI / integration_tests (pull_request) Successful in 10m37s
CI / coverage (pull_request) Successful in 11m31s
CI / status-check (pull_request) Successful in 3s
The `{count:d} semantic chunking fragments should be returned` step
patterns collided with the pre-existing `{count} fragments should be
returned` step in advanced_context_strategies_steps.py:365 — behave's
default `{count}` parser matches `.+?` (non-greedy any char) and
captured "N semantic chunking", failing the registry's ambiguity
check at module-load time. The crash aborted load_step_definitions
for the entire unit_tests session, errored all 8 features in the
behave-parallel worker, and produced the CI failure with verdict
"0 features passed, 0 failed, 8 errored".
Rephrase the two ambiguous step patterns to put unique anchor words
first ("the semantic chunking result should contain {count:d}
fragments" / "...should contain at most {count:d} fragments") and
update the feature file's three call sites to match. Also mark two
defensive private-helper early-return branches with `# pragma: no
cover` — they are unreachable through the public ContextStrategy API
(`_default_embedding("")` is gated by `if not self._anchor` in
`assemble`; `_cosine_similarity` size mismatch is impossible because
all `_get_embedding` callers receive same-length vectors from the
same `embedding_fn`).
Local gates: lint, typecheck, full unit_tests (16 scenarios / 56
steps in the semantic_chunking feature pass; full suite passes),
integration_tests — all green.
ISSUES CLOSED: #9996
133 lines
7.3 KiB
Gherkin
133 lines
7.3 KiB
Gherkin
@phase2 @acms @semantic_chunking
|
|
Feature: SemanticChunkingStrategy — embedding-based context chunking
|
|
As a CleverAgents developer
|
|
I want a SemanticChunkingStrategy that uses embedding similarity
|
|
So that the ACMS pipeline can retain the most semantically relevant
|
|
context chunks for complex multi-turn agent workflows
|
|
|
|
@tdd_issue @tdd_issue_9996
|
|
Scenario: SemanticChunkingStrategy has correct name
|
|
Given a SemanticChunkingStrategy with default parameters
|
|
Then the SemanticChunkingStrategy name should be "semantic_chunking"
|
|
|
|
@tdd_issue @tdd_issue_9996
|
|
Scenario: SemanticChunkingStrategy supports semantic search capability
|
|
Given a SemanticChunkingStrategy with default parameters
|
|
Then the SemanticChunkingStrategy should support semantic search
|
|
|
|
@tdd_issue @tdd_issue_9996
|
|
Scenario: SemanticChunkingStrategy can_handle returns confidence score
|
|
Given a SemanticChunkingStrategy with default parameters
|
|
When I check can_handle on SemanticChunkingStrategy with query "test query"
|
|
Then the SemanticChunkingStrategy confidence should be greater than 0.0
|
|
|
|
@tdd_issue @tdd_issue_9996
|
|
Scenario: SemanticChunkingStrategy can_handle returns lower confidence without query
|
|
Given a SemanticChunkingStrategy with default parameters
|
|
When I check can_handle on SemanticChunkingStrategy without query
|
|
Then the SemanticChunkingStrategy confidence should be 0.1
|
|
|
|
@tdd_issue @tdd_issue_9996
|
|
Scenario: SemanticChunkingStrategy explain returns description
|
|
Given a SemanticChunkingStrategy with default parameters
|
|
Then the SemanticChunkingStrategy explain should contain "semantic"
|
|
|
|
@tdd_issue @tdd_issue_9996
|
|
Scenario: SemanticChunkingStrategy accepts embedding_model parameter
|
|
Given a SemanticChunkingStrategy with embedding_model "text-embedding-ada-002"
|
|
Then the SemanticChunkingStrategy embedding_model should be "text-embedding-ada-002"
|
|
|
|
@tdd_issue @tdd_issue_9996
|
|
Scenario: SemanticChunkingStrategy accepts top_k parameter
|
|
Given a SemanticChunkingStrategy with top_k 5
|
|
Then the SemanticChunkingStrategy top_k should be 5
|
|
|
|
@tdd_issue @tdd_issue_9996
|
|
Scenario: SemanticChunkingStrategy has default top_k of 10
|
|
Given a SemanticChunkingStrategy with default parameters
|
|
Then the SemanticChunkingStrategy top_k should be 10
|
|
|
|
@tdd_issue @tdd_issue_9996
|
|
Scenario: SemanticChunkingStrategy returns empty for empty input
|
|
Given a SemanticChunkingStrategy with default parameters
|
|
And an empty semantic chunking fragment list
|
|
And a semantic chunking budget with max_tokens 1000 and reserved_tokens 0
|
|
When I assemble with the SemanticChunkingStrategy with anchor "test query"
|
|
Then the semantic chunking result should contain 0 fragments
|
|
|
|
@tdd_issue @tdd_issue_9996
|
|
Scenario: SemanticChunkingStrategy ranks fragments by cosine similarity to anchor
|
|
Given a SemanticChunkingStrategy with mock embeddings
|
|
And the following semantic chunking fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/db.py | database connection pool manager | 0.5 | 20 | 3 |
|
|
| project://app/io.py | file input output handler | 0.5 | 15 | 3 |
|
|
| project://app/sql.py | SQL database query executor | 0.5 | 25 | 3 |
|
|
And a semantic chunking budget with max_tokens 1000 and reserved_tokens 0
|
|
When I assemble with the SemanticChunkingStrategy with anchor "database connection"
|
|
Then the first semantic chunking result should have uko_node "project://app/db.py"
|
|
|
|
@tdd_issue @tdd_issue_9996
|
|
Scenario: SemanticChunkingStrategy respects top_k limit
|
|
Given a SemanticChunkingStrategy with top_k 2 and mock embeddings
|
|
And the following semantic chunking fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | alpha beta gamma | 0.5 | 10 | 3 |
|
|
| project://app/b.py | delta epsilon zeta | 0.5 | 10 | 3 |
|
|
| project://app/c.py | eta theta iota | 0.5 | 10 | 3 |
|
|
| project://app/d.py | kappa lambda mu | 0.5 | 10 | 3 |
|
|
And a semantic chunking budget with max_tokens 1000 and reserved_tokens 0
|
|
When I assemble with the SemanticChunkingStrategy with anchor "alpha beta"
|
|
Then the semantic chunking result should contain at most 2 fragments
|
|
|
|
@tdd_issue @tdd_issue_9996
|
|
Scenario: SemanticChunkingStrategy respects token budget
|
|
Given a SemanticChunkingStrategy with default parameters
|
|
And the following semantic chunking fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | hello world | 0.9 | 100 | 3 |
|
|
| project://app/b.py | hello there | 0.7 | 100 | 3 |
|
|
| project://app/c.py | hello again | 0.5 | 100 | 3 |
|
|
And a semantic chunking budget with max_tokens 250 and reserved_tokens 0
|
|
When I assemble with the SemanticChunkingStrategy with anchor "hello"
|
|
Then the semantic chunking result should contain at most 2 fragments
|
|
|
|
@tdd_issue @tdd_issue_9996
|
|
Scenario: SemanticChunkingStrategy falls back to relevance ordering without anchor
|
|
Given a SemanticChunkingStrategy with default parameters
|
|
And the following semantic chunking fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | alpha | 0.3 | 10 | 3 |
|
|
| project://app/b.py | beta | 0.9 | 10 | 3 |
|
|
And a semantic chunking budget with max_tokens 1000 and reserved_tokens 0
|
|
When I assemble with the SemanticChunkingStrategy without anchor
|
|
Then the first semantic chunking result should have uko_node "project://app/b.py"
|
|
|
|
@tdd_issue @tdd_issue_9996
|
|
Scenario: SemanticChunkingStrategy caches embeddings to avoid redundant calls
|
|
Given a SemanticChunkingStrategy with call-counting mock embeddings
|
|
And the following semantic chunking fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | same content | 0.5 | 10 | 3 |
|
|
| project://app/b.py | same content | 0.5 | 10 | 3 |
|
|
And a semantic chunking budget with max_tokens 1000 and reserved_tokens 0
|
|
When I assemble with the SemanticChunkingStrategy with anchor "test"
|
|
Then the embedding model should be called fewer times than total fragments plus anchor
|
|
|
|
@tdd_issue @tdd_issue_9996
|
|
Scenario: SemanticChunkingStrategy is registered in the ACMS pipeline
|
|
Given an ACMS pipeline for semantic chunking tests
|
|
Then the sc_pipeline should have strategy "semantic_chunking"
|
|
|
|
@tdd_issue @tdd_issue_9996
|
|
Scenario: SemanticChunkingStrategy can be used via ACMS pipeline assemble
|
|
Given an ACMS pipeline for semantic chunking tests
|
|
And a valid plan ID for semantic chunking
|
|
And the following semantic chunking fragments:
|
|
| uko_node | content | score | tokens | depth |
|
|
| project://app/a.py | alpha content | 0.8 | 10 | 3 |
|
|
| project://app/b.py | beta content | 0.6 | 10 | 3 |
|
|
And a semantic chunking budget with max_tokens 1000 and reserved_tokens 0
|
|
When I assemble via the pipeline with strategy "semantic_chunking"
|
|
Then the pipeline payload should contain at least 1 fragment
|