UAT: `agents validation attach` blocks duplicate attachments with different args — spec requires allowing them #1418

New issue

Open

opened 2026-04-02 17:45:48 +00:00 by freemo · 0 comments

freemo commented

2026-04-02 17:45:48 +00:00

Owner

Bug Report

Feature Area: Validation Pipeline (v3.4.0)

What Was Tested

The ability to attach the same validation to the same resource+scope combination multiple times with different arguments.

Expected Behavior (from spec)

Per docs/specification.md line 9541:

The same validation can be attached to the same resource multiple times — directly, through different projects, through different plans, and even to the same resource and scope multiple times with different arguments.

And line 22426:

Since validations accept arguments, the same validation can be attached to the same resource multiple times with different arguments. For example, local/run-tests might be attached to local/api-repo through project local/api-service with --coverage-threshold 90 and also through project local/staging with --coverage-threshold 70.

Actual Behavior

The ValidationAttachmentRepository.attach() method raises DuplicateValidationAttachmentError when the same validation+resource+scope combination is used, even with different arguments:

# In repositories.py:
dup_conditions = [
    ValidationAttachmentModel.validation_name == validation_name,
    ValidationAttachmentModel.resource_id == resource_id,
    # project_name and plan_id conditions...
]
existing = session.query(ValidationAttachmentModel).filter(*dup_conditions).first()
if isinstance(existing, ValidationAttachmentModel):
    raise DuplicateValidationAttachmentError(...)  # Does NOT check args_json!

The duplicate check does NOT include args_json in the conditions, so attaching the same validation to the same resource+scope with different arguments (e.g., --coverage-threshold 90 vs --coverage-threshold 70) will fail with a duplicate error.

Steps to Reproduce

# First attachment - succeeds
agents validation attach --project local/api-service local/api-repo local/run-tests --coverage-threshold 90

# Second attachment with different args - should succeed but fails
agents validation attach --project local/api-service local/api-repo local/run-tests --coverage-threshold 70
# Error: DuplicateValidationAttachmentError

Code Location

src/cleveragents/infrastructure/database/repositories.py — ValidationAttachmentRepository.attach() method (~line 3795-3820). The duplicate check needs to include args_json in the uniqueness conditions.

Severity

High — This prevents a core use case explicitly described in the spec: running the same validation with different thresholds for different projects/scopes.

Metadata

Branch: bugfix/m5-validation-attach-duplicate-args
Commit Message: fix(validation): include args_json in duplicate attachment check to allow same validation with different args
Milestone: v3.5.0
Parent Epic: #357

Subtasks

Update ValidationAttachmentRepository.attach() duplicate check to include args_json in uniqueness conditions
Verify the database schema/unique constraint also reflects args_json as part of the uniqueness key (add Alembic migration if needed)
Tests (Behave): Add scenario "Same validation attached twice with different args succeeds"
Tests (Behave): Add scenario "Same validation attached twice with identical args raises DuplicateValidationAttachmentError"
Tests (Robot): Add integration test for multi-arg attachment
Verify coverage >= 97% via nox -s coverage_report
Run nox (all default sessions), fix any errors

Definition of Done

All subtasks are completed and checked off
Attaching the same validation+resource+scope with different args_json succeeds
Attaching the same validation+resource+scope with identical args_json still raises DuplicateValidationAttachmentError
Database uniqueness constraint is consistent with the code logic
BDD scenarios cover both the allowed and rejected cases
A PR is opened from the branch to master, reviewed, and merged
All nox stages pass
Coverage >= 97%

## Bug Report **Feature Area:** Validation Pipeline (v3.4.0) ### What Was Tested The ability to attach the same validation to the same resource+scope combination multiple times with different arguments. ### Expected Behavior (from spec) Per `docs/specification.md` line 9541: > The same validation can be attached to the same resource multiple times — directly, through different projects, through different plans, and even to the same resource and scope multiple times with different arguments. And line 22426: > Since validations accept arguments, the same validation can be attached to the same resource multiple times with different arguments. For example, `local/run-tests` might be attached to `local/api-repo` through project `local/api-service` with `--coverage-threshold 90` and also through project `local/staging` with `--coverage-threshold 70`. ### Actual Behavior The `ValidationAttachmentRepository.attach()` method raises `DuplicateValidationAttachmentError` when the same validation+resource+scope combination is used, even with different arguments: ```python # In repositories.py: dup_conditions = [ ValidationAttachmentModel.validation_name == validation_name, ValidationAttachmentModel.resource_id == resource_id, # project_name and plan_id conditions... ] existing = session.query(ValidationAttachmentModel).filter(*dup_conditions).first() if isinstance(existing, ValidationAttachmentModel): raise DuplicateValidationAttachmentError(...) # Does NOT check args_json! ``` The duplicate check does NOT include `args_json` in the conditions, so attaching the same validation to the same resource+scope with different arguments (e.g., `--coverage-threshold 90` vs `--coverage-threshold 70`) will fail with a duplicate error. ### Steps to Reproduce ```bash # First attachment - succeeds agents validation attach --project local/api-service local/api-repo local/run-tests --coverage-threshold 90 # Second attachment with different args - should succeed but fails agents validation attach --project local/api-service local/api-repo local/run-tests --coverage-threshold 70 # Error: DuplicateValidationAttachmentError ``` ### Code Location `src/cleveragents/infrastructure/database/repositories.py` — `ValidationAttachmentRepository.attach()` method (~line 3795-3820). The duplicate check needs to include `args_json` in the uniqueness conditions. ### Severity High — This prevents a core use case explicitly described in the spec: running the same validation with different thresholds for different projects/scopes. --- ## Metadata - **Branch**: `bugfix/m5-validation-attach-duplicate-args` - **Commit Message**: `fix(validation): include args_json in duplicate attachment check to allow same validation with different args` - **Milestone**: v3.5.0 - **Parent Epic**: #357 ## Subtasks - [ ] Update `ValidationAttachmentRepository.attach()` duplicate check to include `args_json` in uniqueness conditions - [ ] Verify the database schema/unique constraint also reflects `args_json` as part of the uniqueness key (add Alembic migration if needed) - [ ] Tests (Behave): Add scenario "Same validation attached twice with different args succeeds" - [ ] Tests (Behave): Add scenario "Same validation attached twice with identical args raises DuplicateValidationAttachmentError" - [ ] Tests (Robot): Add integration test for multi-arg attachment - [ ] Verify coverage >= 97% via `nox -s coverage_report` - [ ] Run `nox` (all default sessions), fix any errors ## Definition of Done - [ ] All subtasks are completed and checked off - [ ] Attaching the same validation+resource+scope with different `args_json` succeeds - [ ] Attaching the same validation+resource+scope with identical `args_json` still raises `DuplicateValidationAttachmentError` - [ ] Database uniqueness constraint is consistent with the code logic - [ ] BDD scenarios cover both the allowed and rejected cases - [ ] A PR is opened from the branch to `master`, reviewed, and merged - All nox stages pass - Coverage >= 97%

freemo added the

labels

2026-04-02 17:47:23 +00:00

freemo added this to the v3.5.0 milestone