Commit Graph

4 Commits

Author SHA1 Message Date
hurui200320 e2b127b7e5 fix(e2e): replace naive OpenAI key-presence check with live API probe in E2E suite setups
CI / lint (pull_request) Successful in 37s
CI / typecheck (pull_request) Successful in 1m18s
CI / security (pull_request) Successful in 56s
CI / quality (pull_request) Successful in 48s
CI / build (pull_request) Successful in 30s
CI / helm (pull_request) Successful in 41s
CI / push-validation (pull_request) Successful in 28s
CI / integration_tests (pull_request) Successful in 4m32s
CI / e2e_tests (pull_request) Successful in 4m42s
CI / coverage (pull_request) Successful in 13m24s
CI / unit_tests (pull_request) Successful in 3m13s
CI / docker (pull_request) Successful in 1m36s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-regression (push) Failing after 0s
CI / benchmark-publish (push) Failing after 0s
CI / push-validation (push) Successful in 12s
CI / build (push) Successful in 15s
CI / helm (push) Successful in 16s
CI / lint (push) Successful in 43s
CI / typecheck (push) Successful in 51s
CI / security (push) Successful in 51s
CI / e2e_tests (push) Successful in 2m14s
CI / quality (push) Successful in 3m44s
CI / integration_tests (push) Successful in 7m0s
CI / unit_tests (push) Successful in 8m33s
CI / coverage (push) Successful in 6m21s
CI / docker (push) Successful in 1m31s
CI / status-check (push) Successful in 2s
The existing actor-selection logic in several E2E suite setups checked only
whether OPENAI_API_KEY was present (non-empty). A valid key that has hit its
quota limit passes that check but fails at runtime with HTTP 429, causing the
test to fail even though Anthropic credits are available.

Changes:
- Add robot/e2e/check_openai_key.py: stdlib-only (urllib.request) script that
  sends a minimal chat-completion request ('Hi', max_tokens=1, gpt-4o-mini) to
  the OpenAI API. Exits 0 on HTTP 200; exits 1 for quota (429), auth (401),
  network errors, or any other failure.
- Add 'Resolve LLM Actor' keyword to robot/e2e/common_e2e.resource: runs the
  probe script via ${PYTHON} and returns the openai_model argument (default
  openai/gpt-4o) on success, or the anthropic_model argument (default
  anthropic/claude-sonnet-4-20250514) on failure. Skips the probe entirely when
  OPENAI_API_KEY is not set.
- Update m6_acceptance.robot, wf04_multi_project.robot, wf05_db_migration.robot,
  wf07_cicd.robot, and wf16_devcontainer.robot to use 'Resolve LLM Actor'
  instead of the inline has_openai boolean check.

No production source code (src/) is modified. The decision to fall back to
Anthropic is made once per suite setup, before any test runs.

Closes #10198
2026-04-17 18:00:47 +08:00
freemo 8ea00f5185 fix: restore CI quality tests to passing state (#4175)
CI / unit_tests (push) Has been cancelled
CI / benchmark-publish (push) Has been cancelled
CI / lint (push) Has been cancelled
CI / typecheck (push) Has been cancelled
CI / security (push) Has been cancelled
CI / quality (push) Has been cancelled
CI / integration_tests (push) Has been cancelled
CI / e2e_tests (push) Has been cancelled
CI / coverage (push) Has been cancelled
CI / benchmark-regression (push) Has been cancelled
CI / build (push) Has been cancelled
CI / push-validation (push) Has been cancelled
CI / status-check (push) Has been cancelled
CI / docker (push) Has been cancelled
CI / helm (push) Has been cancelled
Co-authored-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
Co-committed-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
2026-04-08 11:02:14 +00:00
freemo 48cff5cfe0 refactor(cli): rename plan lifecycle-list and lifecycle-apply to match specification
CI / build (push) Successful in 18s
CI / lint (push) Failing after 31s
CI / helm (push) Successful in 33s
CI / typecheck (push) Successful in 50s
CI / security (push) Failing after 51s
CI / coverage (push) Has been skipped
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Failing after 1m50s
CI / docker (push) Has been skipped
CI / quality (push) Successful in 3m43s
CI / integration_tests (push) Has been cancelled
CI / e2e_tests (push) Has been cancelled
CI / benchmark-publish (push) Has been cancelled
CI / status-check (push) Has been cancelled
Renames `plan lifecycle-list` to `plan list` and `plan lifecycle-apply` to `plan apply` to align with the specification's canonical command names. Removes legacy V2 plan commands that occupied those names.

- Renamed CLI command registrations from lifecycle-list/lifecycle-apply to list/apply
- Removed legacy V2 apply and list commands (~200 lines)
- Updated apply shortcut in main.py to delegate to v3 lifecycle
- Added defensive null check for plan existence in apply command
- Updated 63+ test, doc, and benchmark files for consistency

Closes #881

Co-authored-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
Co-committed-by: Jeffrey Phillips Freeman <the@jeffreyfreeman.me>
2026-04-02 19:09:04 +00:00
freemo 845cf61b47 test(e2e): workflow example 4 — multi-project dependency update (supervised profile)
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 38s
CI / security (pull_request) Successful in 1m17s
CI / build (pull_request) Successful in 25s
CI / helm (pull_request) Successful in 37s
CI / quality (pull_request) Successful in 3m43s
CI / typecheck (pull_request) Successful in 3m58s
CI / integration_tests (pull_request) Successful in 6m4s
CI / unit_tests (pull_request) Successful in 6m22s
CI / docker (pull_request) Successful in 21s
CI / e2e_tests (pull_request) Successful in 9m29s
CI / coverage (pull_request) Successful in 9m12s
CI / status-check (pull_request) Successful in 1s
CI / lint (push) Successful in 31s
CI / security (push) Successful in 55s
CI / build (push) Successful in 21s
CI / helm (push) Successful in 22s
CI / quality (push) Successful in 3m45s
CI / typecheck (push) Successful in 3m56s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 6m46s
CI / integration_tests (push) Successful in 6m25s
CI / docker (push) Successful in 1m20s
CI / e2e_tests (push) Successful in 9m57s
CI / coverage (push) Successful in 10m50s
CI / status-check (push) Successful in 1s
CI / benchmark-publish (push) Successful in 29m50s
CI / benchmark-regression (pull_request) Successful in 56m29s
Implement WF04 E2E test coverage for multi-project dependency update
using the supervised automation profile. A parent plan targets four
projects (common-lib + 3 services), spawns child subplans per project,
executes in dependency order, validates per project, and applies in
dependency order.

Key components:
- Robot test (robot/e2e/wf04_multi_project.robot): full supervised
  workflow exercising plan use, strategize, execute, validate, apply,
  and final status. JSON output parsing via raw_decode for resilience.
  Test-level guard skips the entire test (visible SKIPPED in CI) when
  the LLM produces zero subplans, preventing silent AC-4/5/6/7 bypass.
  Count Decision Nodes keyword delegates to the Python helper to avoid
  duplicating recursive tree-walking logic.
- Snapshot helper (robot/e2e/wf04_snapshot_helper.py): collects
  deterministic parent/subplan metadata from the lifecycle service.
  Timestamps normalised to UTC. Depth-guarded count_decision_nodes()
  function (max_depth=50) for safe reuse from Robot keywords.
  sys.path.append (not insert) to avoid shadowing the standard library.
- Behave unit tests (features/wf04_snapshot_helper.feature + steps):
  17 scenarios covering _iso(), _enum_value(), count_decision_nodes(),
  and _build_snapshot() with mocked lifecycle service. Includes tests
  for no-subplans (verifies plan_id, project_scopes, validation_summary),
  with-subplans (verifies concrete mapped project names and subplan_count),
  child-plan-is-None (verifies empty-string defaults), unmapped resources,
  and exact-match aware-datetime UTC normalisation.

ISSUES CLOSED: #750
2026-03-30 11:47:37 +08:00