5 Commits

Author SHA1 Message Date
HAL9000 de1fd49594 fix(plan-lifecycle): fix integration test regressions from DoD gating
Update integration test helpers to use definition_of_done values that
satisfy the new TextMatchEvaluator gate (criterion text must appear as
a substring of a plan argument key or value).  Also fix _evaluate_dod
to merge DoD metadata into plan.validation_summary without overwriting
Execute-phase validation counts, preserving the coverage/tool-validation
gate used by apply_with_validation_gate.  Update CONTRIBUTORS.md.

Fixes: wf02, wf04, wf06, wf07 integration test suites.

ISSUES CLOSED: #7927
2026-06-02 05:14:43 -04:00
CoreRasurae b41f536da6 fix(automation): respect automation profile gates in lifecycle service and async jobs
CI / helm (push) Successful in 47s
CI / build (push) Successful in 1m11s
CI / push-validation (push) Successful in 40s
CI / benchmark-publish (push) Has started running
CI / lint (push) Successful in 1m26s
CI / typecheck (push) Successful in 1m55s
CI / e2e_tests (push) Successful in 1m12s
CI / quality (push) Successful in 2m6s
CI / security (push) Successful in 2m9s
CI / benchmark-regression (push) Failing after 1m28s
CI / integration_tests (push) Successful in 3m48s
CI / unit_tests (push) Successful in 7m31s
CI / coverage (push) Has started running
CI / docker (push) Successful in 1m35s
CI / helm (pull_request) Successful in 39s
CI / push-validation (pull_request) Successful in 39s
CI / build (pull_request) Successful in 1m13s
CI / lint (pull_request) Successful in 1m32s
CI / quality (pull_request) Successful in 1m49s
CI / typecheck (pull_request) Successful in 1m48s
CI / security (pull_request) Successful in 1m58s
CI / status-check (push) Has been cancelled
CI / integration_tests (pull_request) Successful in 4m48s
CI / unit_tests (pull_request) Successful in 5m11s
CI / coverage (pull_request) Has started running
CI / docker (pull_request) Successful in 1m44s
CI / status-check (pull_request) Has been cancelled
Add BDD regression tests for issue #4328 automation profile gates

Tests verify that complete_strategize() and complete_execute() respect
automation profile thresholds and do NOT unconditionally call auto_progress().

Covers 8 built-in profiles: manual, full-auto, supervised, auto,
review_before_apply, ci, trusted

ISSUES CLOSED: #4328
2026-05-14 22:57:11 +00:00
freemo 1d36449a98 fix(cli): remove extra --mode flag from validation attach
CI / build (push) Has been cancelled
CI / e2e_tests (push) Has been cancelled
CI / typecheck (push) Has been cancelled
CI / integration_tests (push) Has been cancelled
CI / quality (push) Has been cancelled
CI / lint (push) Has been cancelled
CI / benchmark-publish (push) Has been cancelled
CI / security (push) Has been cancelled
CI / helm (push) Has been cancelled
CI / unit_tests (push) Has been cancelled
CI / benchmark-regression (push) Has been cancelled
CI / coverage (push) Has been cancelled
CI / docker (push) Has been cancelled
CI / status-check (push) Has been cancelled
Remove the --mode/-m CLI flag from the validation attach command to align
with the specification, which defines validation mode as an inherent
property of the validation definition set at registration time via
validation add, not as a per-attachment override.

The service layer (ToolRegistryService.attach_validation) no longer
accepts mode as a parameter; instead it reads the mode from the
validation's registered definition. The ToolRegistryRepository
_to_legacy_domain now exposes the mode field so the service can access it.

All tests that passed --mode to the CLI or service have been updated or
removed. The invalid-mode validation scenarios were removed since the
mode is no longer caller-supplied. Coverage remains at 98.7%.

ISSUES CLOSED: #913

Co-authored-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com>
Co-committed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com>
2026-04-02 17:07:26 +00:00
brent.edwards 34dcf5226f feat(autonomy): automation profile resolution precedence correct (#1196)
CI / build (push) Successful in 18s
CI / helm (push) Successful in 38s
CI / lint (push) Successful in 3m19s
CI / quality (push) Successful in 3m42s
CI / typecheck (push) Successful in 3m56s
CI / security (push) Successful in 4m7s
CI / unit_tests (push) Failing after 5m43s
CI / docker (push) Has been skipped
CI / coverage (push) Successful in 12m28s
CI / e2e_tests (push) Successful in 20m5s
CI / integration_tests (push) Failing after 39m29s
CI / status-check (push) Failing after 8m55s
CI / benchmark-regression (push) Has been skipped
CI / benchmark-publish (push) Successful in 28m30s
## Summary
- Fix automation profile resolution precedence at plan-creation time to follow plan > action > project > global.
- Narrow exception handling in `_resolve_plan_profile_ref` to catch only `(KeyError, ValueError, OSError)` with warning-level structured logging, replacing bare `except Exception:` per CONTRIBUTING.md §Exception Propagation.
- Fix TDD feature file tag/title inconsistency: align to `@tdd_bug @tdd_bug_1076` / "TDD Bug #1076" referencing the original bug ticket.
- Add BDD scenarios for plan-level profile override and config service error fallback.
- Harden lifecycle and Robot/Behave test paths for auto-progression and parallel CI stability (including pre-seeded DB initialization for `tdd_plan_explain_plan_id` helper).
- Update Robot helper timing/session patterns to reduce false negatives in long-running integration/E2E validations.

## Approach
The automation profile resolution is implemented in `PlanLifecycleService._resolve_plan_profile_ref`, which evaluates the four-level precedence chain (plan > action > project > global) at `use_action()` time. The resolved profile is stored as an `AutomationProfileRef` on the Plan with provenance tracking. The `ConfigService` is used for project-scoped and global config lookups, with narrowed exception handling that logs warnings on failure and falls back to the settings default.

## Validation
- `nox -s lint` 
- `nox -s typecheck` 
- `nox -s unit_tests` 
- `nox -s coverage_report`  (97%)

## Review Fix Round (Review #2963)
1. **Bare `except Exception:` → narrowed to `(KeyError, ValueError, OSError)` with warning logging** — addresses CONTRIBUTING.md §Exception Propagation violation
2. **Tag inconsistency fixed** — `@tdd_bug @tdd_bug_1076` with title "TDD Bug #1076"
3. **Added 2 new BDD scenarios** — plan-level override and config error fallback
4. **Rebased onto latest master** (532ea100)

Closes #854

Reviewed-on: #1196
Co-authored-by: Brent Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent Edwards <brent.edwards@cleverthis.com>
2026-04-01 04:27:09 +00:00
CoreRasurae 80f1664a0d test(integration): workflow example 7 — CI/CD integration, automated PR review and fix (ci profile)
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 23s
CI / lint (pull_request) Successful in 3m20s
CI / typecheck (pull_request) Successful in 3m54s
CI / security (pull_request) Successful in 4m13s
CI / quality (pull_request) Successful in 4m17s
CI / integration_tests (pull_request) Successful in 9m21s
CI / unit_tests (pull_request) Successful in 9m42s
CI / docker (pull_request) Successful in 14s
CI / e2e_tests (pull_request) Successful in 12m1s
CI / coverage (pull_request) Successful in 11m41s
CI / status-check (pull_request) Successful in 1s
CI / build (push) Successful in 26s
CI / lint (push) Successful in 3m19s
CI / quality (push) Successful in 3m41s
CI / typecheck (push) Successful in 3m56s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 4m2s
CI / integration_tests (push) Successful in 9m3s
CI / unit_tests (push) Successful in 9m24s
CI / docker (push) Successful in 1m9s
CI / e2e_tests (push) Successful in 13m51s
CI / coverage (push) Successful in 11m25s
CI / status-check (push) Successful in 1s
CI / benchmark-publish (push) Failing after 24m52s
CI / benchmark-regression (pull_request) Failing after 37m6s
Implemented Robot Framework integration test suite validating the CI/CD
workflow (Specification Example 7). Tests cover ci-profile configuration
(automation-profile, format, log level), idempotent resource and project
registration with duplicate-detection assertions, three validation tools
(ci-lint, ci-typecheck, ci-tests) registration and resource attachment
via ToolRegistryService, action creation with typed arguments and
invariants per spec Step 2, plan lifecycle with phase-by-phase completion
through all phases (strategize, execute, apply) until terminal applied
state, and JSON output structure verification including plan_id, phase,
state, action, projects, and arguments fields.

Post-review fixes applied (round 1):
- H1: Fix truncated invariant #2 text to match spec line 39051
- H2: Fix definition_of_done to match spec's 5-bullet-point format
- M1: Register all 3 spec validations (ci-lint, ci-typecheck, ci-tests)
- M3: Add branch property to resource registration per spec Step 3
- M4: Replace phantom resource_id with real registered resource
- M5: Add projects and arguments field assertions to JSON output test
- M7: Replace 'polling-based' with 'phase-by-phase' in docs/comments
- L1: Use timezone-aware datetime (timezone.utc)
- L2: Use tempfile for resource path instead of hardcoded /tmp
- L3: Clarify json_output() docstring to reflect as_cli_dict() scope
- L4: Tighten attachment count assertion from >= 1 to == 1 (now == 3)

Post-review fixes applied (round 2):
- M2: Use Setup Test Environment With Database Isolation for pabot safety
- L1: Replace remaining hardcoded /tmp paths with tempfile (validation_attach,
  resource_idempotent duplicate registration)
- L2: Fix stale 'Polling-based' comment missed in round 1
- L5: Align action description with spec line 39027
- M1/L3/L6: Document automation_profile propagation gap in use_action()
  with TODO for when production code wires the field onto Plan
- L4: Add comment explaining local actor name deviation from spec

Includes CHANGELOG update describing the integration test scope.

ISSUES CLOSED: #771
2026-03-26 21:42:37 +00:00