brent.edwards
81c141716b
CI / build (push) Successful in 22s
CI / helm (push) Successful in 22s
CI / lint (push) Successful in 3m20s
CI / quality (push) Successful in 3m46s
CI / typecheck (push) Successful in 3m56s
CI / security (push) Successful in 4m11s
CI / unit_tests (push) Successful in 8m55s
CI / docker (push) Successful in 1m35s
CI / coverage (push) Successful in 12m29s
CI / e2e_tests (push) Successful in 18m50s
CI / integration_tests (push) Successful in 24m10s
CI / status-check (push) Successful in 1s
CI / benchmark-publish (push) Has been cancelled
CI / benchmark-regression (push) Has been cancelled
## Summary
- Refactors WF02 integration helper into focused modules to satisfy maintainability guidelines and keep command responsibilities isolated.
- Moves WF02 artifact assertions onto lifecycle-produced change artifacts and verifies user-facing `plan artifacts` flow with mocked providers.
- Reworks WF02 coverage-gate assertions to run through validation registration/attachment/execution lifecycle and apply-gate outcomes.
## What changed in this fix round (cycle-6)
- **CI must-fix — trusted profile lifecycle integration failure:**
- Fixed failing integration test `WF02 Trusted Profile Auto Runs Strategize And Execute` caused by legacy AutomationProfile fields that no longer exist.
- Updated WF02 trusted-profile assertions from removed legacy fields (`auto_strategize`, `auto_execute`, `auto_apply`) to current spec-aligned fields (`decompose_task`, `create_tool`, `select_tool`).
- Key module: `robot/wf02_test_generation_commands.py` (`wf02_trusted_lifecycle`).
## Prior fix rounds retained
- **P1 must-fix — validation payload assertions strengthened:**
- Added explicit assertions for validation payload correctness in both failing and passing branches.
- Assertions now verify:
- validation identity (`local/auth-coverage` on `local/api-service-repo`)
- required-mode semantics (`mode=required`, required pass/fail counters, `all_required_passed`)
- `passed` boolean for each branch
- measured-vs-target semantics (`measured_coverage` compared against `target`).
- Key module: `robot/wf02_test_generation_validation.py`.
- **P2 should-fix — artifact membership assertions tightened:**
- Artifact assertions now enforce exact expected membership and count instead of permissive subset checks.
- Persisted changeset assertions were aligned to the same strict set/count checks.
- Key module: `robot/wf02_test_generation_artifacts.py`.
- **P2 should-fix — removed unused helper:**
- Deleted unused `_bind_changeset_to_plan` helper to eliminate dead code.
- Key module: `robot/wf02_test_generation_artifacts.py`.
- **P2 should-fix — concrete typing:**
- Replaced `_setup_validation_db` return type from `tuple[Any, Any]` to concrete typed return (`Callable[[], _NoCloseSession]`, `Session`).
- Key module: `robot/wf02_test_generation_validation.py`.
## Quality gates
- `nox -e lint` ✅
- `nox -e typecheck` ✅
- `nox -e unit_tests` ✅
- `nox -e integration_tests` ✅
- `nox -e e2e_tests` ✅
- `nox -e coverage_report` ✅ (coverage: **98.74%**)
## Scope / limitations
- No deferred items identified in cycle-6.
Closes #766
Reviewed-on: #1176
Reviewed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com>
Co-authored-by: Brent Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent Edwards <brent.edwards@cleverthis.com>
48 lines
1.4 KiB
Python
48 lines
1.4 KiB
Python
"""WF02 helper command dispatcher for Robot integration tests."""
|
|
|
|
from __future__ import annotations
|
|
|
|
import sys
|
|
from collections.abc import Callable
|
|
from pathlib import Path
|
|
|
|
_SRC = str(Path(__file__).resolve().parents[1] / "src")
|
|
if _SRC not in sys.path:
|
|
sys.path.insert(0, _SRC)
|
|
|
|
_ROBOT = str(Path(__file__).resolve().parent)
|
|
if _ROBOT not in sys.path:
|
|
sys.path.insert(0, _ROBOT)
|
|
|
|
from helpers_common import reset_global_state # noqa: E402
|
|
from wf02_test_generation_artifacts import ( # noqa: E402
|
|
wf02_mocked_generation_artifacts,
|
|
)
|
|
from wf02_test_generation_commands import command_map # noqa: E402
|
|
from wf02_test_generation_validation import wf02_coverage_validation # noqa: E402
|
|
|
|
COMMANDS: dict[str, Callable[[], None]] = command_map(
|
|
mocked_generation_artifacts=wf02_mocked_generation_artifacts,
|
|
coverage_validation=wf02_coverage_validation,
|
|
)
|
|
|
|
|
|
def main() -> int:
|
|
"""Command dispatcher used by Robot ``Run Process`` calls."""
|
|
if len(sys.argv) < 2 or sys.argv[1] not in COMMANDS:
|
|
print(
|
|
"Usage: helper_wf02_test_generation.py "
|
|
"<trusted-lifecycle|mocked-generation-artifacts|coverage-validation"
|
|
"|path-safety-guardrails>",
|
|
file=sys.stderr,
|
|
)
|
|
return 1
|
|
|
|
reset_global_state()
|
|
COMMANDS[sys.argv[1]]()
|
|
return 0
|
|
|
|
|
|
if __name__ == "__main__":
|
|
raise SystemExit(main())
|