UAT: `FAISSVectorBackend.similarity_search()` does not pass `project` parameter — ACMS vector search returns results across all projects #4014

New issue

Open

opened 2026-04-06 08:37:28 +00:00 by freemo · 0 comments

freemo commented

2026-04-06 08:37:28 +00:00

Owner

Metadata

Branch: fix/faiss-vector-backend-project-scoping
Commit Message: fix(acms): pass project parameter in FAISSVectorBackend.similarity_search() to scope vector search results
Milestone: (none — backlog)
Parent Epic: #396

Background and Context

The FAISSVectorBackend class in src/cleveragents/application/services/faiss_vector_backend.py is the read-side ACMS vector backend used by context strategies to perform similarity search. It wraps VectorStoreService.acms_search_by_vector(), which accepts an optional project parameter to scope results to a specific project.

However, FAISSVectorBackend.similarity_search() does not pass the project parameter to acms_search_by_vector(), causing vector similarity search to return results from all projects in the shared FAISS store.

Current Behavior

# src/cleveragents/application/services/faiss_vector_backend.py lines 38-71
class FAISSVectorBackend:
    def similarity_search(
        self,
        embedding: list[float],
        *,
        scope: frozenset[str],
        top_k: int = 20,
    ) -> list[VectorResult]:
        hits = self._vector_store_service.acms_search_by_vector(
            list(embedding),
            top_k=top_k,
            scope=scope,
            # project parameter NOT passed — searches ALL projects
        )

Compare with FAISSVectorIndexBackend.search_similar() (write-side, lines 105-146) which correctly passes project:

# src/cleveragents/application/services/faiss_vector_backend.py lines 105-146
class FAISSVectorIndexBackend:
    def search_similar(
        self,
        project: str,
        query_embedding: list[float],
        ...
    ) -> list[SearchResult]:
        hits = self._vector_store_service.acms_search_by_vector(
            list(query_embedding),
            top_k=limit,
            project=normalized_project,  # project IS passed here
        )

The VectorStoreService.acms_search_by_vector() signature (lines 264-313) accepts project: str | None = None — when None, it searches across all projects.

Expected Behavior

The FAISSVectorBackend.similarity_search() method should accept a project parameter and pass it to acms_search_by_vector() to scope results to the current project. This is consistent with:

The VectorBackend protocol's scope parameter (which scopes by resource ULID)
The FAISSVectorIndexBackend.search_similar() method which correctly uses project
The spec's requirement for project-scoped context assembly

Without project scoping, context strategies using the vector backend may return fragments from other projects' indexed resources, violating project isolation.

Steps to Reproduce

Open src/cleveragents/application/services/faiss_vector_backend.py
Navigate to FAISSVectorBackend.similarity_search() (lines 38-71)
Observe that acms_search_by_vector() is called without project=
Compare with FAISSVectorIndexBackend.search_similar() (lines 105-146) which passes project=normalized_project
Open src/cleveragents/application/services/vector_store_service.py lines 264-313
Observe that acms_search_by_vector() with project=None searches all projects

Impact

When multiple projects have indexed resources in the shared FAISS store, a context strategy query for project A may return results from project B's indexed resources. This violates project isolation and could expose sensitive code or documentation from one project to another project's context assembly.

Code locations affected:

src/cleveragents/application/services/faiss_vector_backend.py lines 38-71 (FAISSVectorBackend.similarity_search() — missing project parameter)
src/cleveragents/application/services/vector_store_service.py lines 264-313 (acms_search_by_vector() — project=None searches all projects)

Backlog note: This issue was discovered during autonomous UAT operation on milestone v3.6.0. It does not block milestone completion and has been placed in the backlog for human review and future milestone assignment.

Subtasks

Add project: str | None = None parameter to FAISSVectorBackend.similarity_search()
Pass project to VectorStoreService.acms_search_by_vector() in FAISSVectorBackend.similarity_search()
Update the VectorBackend protocol in backends.py to include project parameter if needed
Write BDD tests verifying that vector search results are scoped to the correct project
Verify nox -e coverage_report reports >= 97% coverage

Definition of Done

FAISSVectorBackend.similarity_search() passes project to acms_search_by_vector()
Vector search results are scoped to the current project
Multi-project isolation is verified by BDD tests
All BDD unit tests pass
All nox stages pass
Coverage >= 97%

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: ca-new-issue-creator

## Metadata - **Branch**: `fix/faiss-vector-backend-project-scoping` - **Commit Message**: `fix(acms): pass project parameter in FAISSVectorBackend.similarity_search() to scope vector search results` - **Milestone**: *(none — backlog)* - **Parent Epic**: #396 ## Background and Context The `FAISSVectorBackend` class in `src/cleveragents/application/services/faiss_vector_backend.py` is the read-side ACMS vector backend used by context strategies to perform similarity search. It wraps `VectorStoreService.acms_search_by_vector()`, which accepts an optional `project` parameter to scope results to a specific project. However, `FAISSVectorBackend.similarity_search()` does **not** pass the `project` parameter to `acms_search_by_vector()`, causing vector similarity search to return results from **all projects** in the shared FAISS store. ## Current Behavior ```python # src/cleveragents/application/services/faiss_vector_backend.py lines 38-71 class FAISSVectorBackend: def similarity_search( self, embedding: list[float], *, scope: frozenset[str], top_k: int = 20, ) -> list[VectorResult]: hits = self._vector_store_service.acms_search_by_vector( list(embedding), top_k=top_k, scope=scope, # project parameter NOT passed — searches ALL projects ) ``` Compare with `FAISSVectorIndexBackend.search_similar()` (write-side, lines 105-146) which correctly passes `project`: ```python # src/cleveragents/application/services/faiss_vector_backend.py lines 105-146 class FAISSVectorIndexBackend: def search_similar( self, project: str, query_embedding: list[float], ... ) -> list[SearchResult]: hits = self._vector_store_service.acms_search_by_vector( list(query_embedding), top_k=limit, project=normalized_project, # project IS passed here ) ``` The `VectorStoreService.acms_search_by_vector()` signature (lines 264-313) accepts `project: str | None = None` — when `None`, it searches across all projects. ## Expected Behavior The `FAISSVectorBackend.similarity_search()` method should accept a `project` parameter and pass it to `acms_search_by_vector()` to scope results to the current project. This is consistent with: 1. The `VectorBackend` protocol's `scope` parameter (which scopes by resource ULID) 2. The `FAISSVectorIndexBackend.search_similar()` method which correctly uses `project` 3. The spec's requirement for project-scoped context assembly Without project scoping, context strategies using the vector backend may return fragments from other projects' indexed resources, violating project isolation. ## Steps to Reproduce 1. Open `src/cleveragents/application/services/faiss_vector_backend.py` 2. Navigate to `FAISSVectorBackend.similarity_search()` (lines 38-71) 3. Observe that `acms_search_by_vector()` is called without `project=` 4. Compare with `FAISSVectorIndexBackend.search_similar()` (lines 105-146) which passes `project=normalized_project` 5. Open `src/cleveragents/application/services/vector_store_service.py` lines 264-313 6. Observe that `acms_search_by_vector()` with `project=None` searches all projects ## Impact When multiple projects have indexed resources in the shared FAISS store, a context strategy query for project A may return results from project B's indexed resources. This violates project isolation and could expose sensitive code or documentation from one project to another project's context assembly. **Code locations affected:** - `src/cleveragents/application/services/faiss_vector_backend.py` lines 38-71 (`FAISSVectorBackend.similarity_search()` — missing `project` parameter) - `src/cleveragents/application/services/vector_store_service.py` lines 264-313 (`acms_search_by_vector()` — `project=None` searches all projects) > **Backlog note:** This issue was discovered during autonomous UAT operation on milestone v3.6.0. It does not block milestone completion and has been placed in the backlog for human review and future milestone assignment. ## Subtasks - [ ] Add `project: str | None = None` parameter to `FAISSVectorBackend.similarity_search()` - [ ] Pass `project` to `VectorStoreService.acms_search_by_vector()` in `FAISSVectorBackend.similarity_search()` - [ ] Update the `VectorBackend` protocol in `backends.py` to include `project` parameter if needed - [ ] Write BDD tests verifying that vector search results are scoped to the correct project - [ ] Verify `nox -e coverage_report` reports >= 97% coverage ## Definition of Done - [ ] `FAISSVectorBackend.similarity_search()` passes `project` to `acms_search_by_vector()` - [ ] Vector search results are scoped to the current project - [ ] Multi-project isolation is verified by BDD tests - [ ] All BDD unit tests pass - [ ] All nox stages pass - [ ] Coverage >= 97% --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: ca-new-issue-creator

freemo added a new dependency

2026-04-06 08:37:35 +00:00

#396 Epic: ACMS Context Pipeline

freemo referenced this issue

2026-04-06 08:38:21 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo referenced this issue

2026-04-06 08:42:28 +00:00

[Automated] CleverAgents Build Session - 2026-04-06 #3775

freemo added the

Points

label