Commit Graph

6 Commits

Author SHA1 Message Date
HAL9000 80bc9c552d test: restore and enhance e2e test coverage
CI / benchmark-publish (pull_request) Has been skipped
CI / push-validation (pull_request) Successful in 33s
CI / helm (pull_request) Successful in 45s
CI / build (pull_request) Successful in 1m4s
CI / lint (pull_request) Successful in 1m13s
CI / typecheck (pull_request) Successful in 1m31s
CI / quality (pull_request) Successful in 1m37s
CI / security (pull_request) Successful in 1m49s
CI / integration_tests (pull_request) Successful in 3m34s
CI / e2e_tests (pull_request) Successful in 5m6s
CI / unit_tests (pull_request) Successful in 6m14s
CI / docker (pull_request) Successful in 1m36s
CI / coverage (pull_request) Successful in 12m27s
CI / status-check (push) Blocked by required conditions
CI / coverage (push) Blocked by required conditions
CI / docker (push) Blocked by required conditions
CI / unit_tests (push) Has started running
CI / status-check (pull_request) Successful in 4s
CI / helm (push) Successful in 31s
CI / build (push) Successful in 53s
CI / lint (push) Successful in 1m3s
CI / quality (push) Successful in 1m13s
CI / typecheck (push) Successful in 1m46s
CI / security (push) Successful in 1m46s
CI / push-validation (push) Successful in 20s
CI / benchmark-publish (push) Failing after 43s
CI / e2e_tests (push) Successful in 4m9s
CI / integration_tests (push) Successful in 6m57s
Fix malformed imports in helper_m1_e2e_verification.py and
helper_m4_e2e_cli.py where 'from helpers_common import reset_global_state'
was incorrectly inserted inside a parenthesized import block, causing
Python syntax errors and breaking the lint and unit_tests CI gates.

Also fix import ordering in helper_m2_e2e_verification.py,
helper_m5_e2e_context.py, helper_m5_e2e_support.py, and
helper_m5_e2e_verification.py to satisfy ruff I001 import-sort rules
by placing helpers_common imports in the correct group alongside other
local robot/ helper imports.

ISSUES CLOSED: #8459
2026-05-01 00:59:02 +00:00
HAL9000 7aa50ac489 fix(e2e): restore M5/WF14 tests broken by JSON envelope and stale config
- Add NO_COLOR=1 to E2E Suite Setup and Test Environment to prevent
  Rich from injecting ANSI escape codes into JSON output, which breaks
  Extract JSON From Stdout and JSON assertions in M5/WF14 acceptance tests.

- Add reset_global_state() calls to all M1/M2/M4/M5 E2E verification
  helpers to clear Settings singleton, DI container, provider registry,
  and SQLAlchemy engine cache between parallel pabot worker runs.

- Propagate NO_COLOR=1 at suite-setup level in E2E and integration
  test resource files so all Run Process subprocesses inherit clean
  terminal output mode.

Root causes:
1. Missing NO_COLOR=1 caused Rich ANSI codes in CLI JSON output,
   making json.loads() failures in robot Extract JSON From Stdout
2. Stale Settings singleton carried provider keys and DB paths between
   pabot workers, causing UNIQUE constraint violations and stale config
   lookups in context policy and server mode E2E tests.

Related: PR #9912
2026-05-01 00:59:02 +00:00
aditya 4e64544aae feat(acms): projects with 10,000+ files index without timeout
CI / benchmark-publish (pull_request) Has been skipped
CI / quality (pull_request) Successful in 48s
CI / build (pull_request) Successful in 31s
CI / helm (pull_request) Successful in 21s
CI / lint (pull_request) Successful in 3m15s
CI / typecheck (pull_request) Successful in 4m4s
CI / security (pull_request) Successful in 4m13s
CI / unit_tests (pull_request) Successful in 4m28s
CI / docker (pull_request) Successful in 1m52s
CI / integration_tests (pull_request) Successful in 7m14s
CI / coverage (pull_request) Successful in 12m49s
CI / e2e_tests (pull_request) Successful in 22m8s
CI / status-check (pull_request) Successful in 2s
CI / benchmark-regression (pull_request) Successful in 57m27s
Add explicit indexing timeout bounds and propagate timeout_seconds through the repo indexing service, utility walker, and CLI entrypoint so large repositories fail predictably instead of hanging.

Add 10K-scale Behave/Robot verification and a dedicated benchmark path to validate that large-project indexing completes, persists status correctly, and remains measurable for regressions.

Address review feedback by splitting oversized Behave and Robot support files into focused modules, trimming repo_indexing_service.py back under the 500-line limit, and rebasing the branch onto current master.

ISSUES CLOSED: #851

# Conflicts:
#	CHANGELOG.md
#	robot/helper_m5_e2e_verification.py
2026-03-30 10:18:19 +00:00
aditya f0442e835d feat(acms): plan execution leverages ACMS context for LLM calls
CI / typecheck (pull_request) Successful in 50s
CI / lint (pull_request) Successful in 3m31s
CI / security (pull_request) Successful in 4m14s
CI / build (pull_request) Successful in 26s
CI / quality (pull_request) Successful in 3m45s
CI / helm (pull_request) Successful in 40s
CI / integration_tests (pull_request) Successful in 3m57s
CI / unit_tests (pull_request) Successful in 4m24s
CI / docker (pull_request) Successful in 1m29s
CI / coverage (pull_request) Successful in 10m54s
CI / benchmark-publish (pull_request) Has been skipped
CI / e2e_tests (pull_request) Successful in 15m35s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-regression (pull_request) Successful in 1h0m6s
Integrate ACMS execute-phase context assembly into LLMExecuteActor and inject assembled context into execute prompts with resilient fallback when assembly fails or returns empty output.

Wire the plan CLI executor to use an ACMS-backed execute context assembler, add Behave coverage for context injection/fallback/empty context, and extend Robot M5 verification helpers to assert execute-phase ACMS context usage.

ISSUES CLOSED: #850
2026-03-30 09:21:51 +00:00
aditya ef8f9640fa feat(acms): context analysis produces meaningful summaries
CI / lint (pull_request) Successful in 30s
CI / typecheck (pull_request) Successful in 1m7s
CI / quality (pull_request) Successful in 46s
CI / security (pull_request) Successful in 1m0s
CI / build (pull_request) Successful in 23s
CI / helm (pull_request) Successful in 41s
CI / integration_tests (pull_request) Successful in 4m11s
CI / unit_tests (pull_request) Successful in 4m38s
CI / docker (pull_request) Successful in 1m22s
CI / coverage (pull_request) Successful in 11m26s
CI / e2e_tests (pull_request) Successful in 18m21s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 58m2s
Add phase-wise context analysis summaries for strategize, execute, and apply, and integrate them into `agents project context inspect` and `agents project context simulate` with per-phase resource counts, size and token utilization, and narrowing diagnostics.\n\nAdd Behave and Robot coverage for empty, single-resource, multi-resource, and budget-constrained context analysis, and tighten the M5 smoke coverage after review feedback with exact summary assertions and a missing-path regression scenario for `agents context add`.\n\nISSUES CLOSED: #849
2026-03-30 08:42:48 +00:00
brent.edwards fd1612c38f test(e2e): verify M5 success criteria — ACMS and large-project context
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 17s
CI / lint (pull_request) Successful in 21s
CI / quality (pull_request) Successful in 20s
CI / security (pull_request) Successful in 54s
CI / typecheck (pull_request) Successful in 56s
CI / integration_tests (pull_request) Successful in 4m21s
CI / unit_tests (pull_request) Successful in 10m21s
CI / docker (pull_request) Successful in 39s
CI / benchmark-regression (pull_request) Successful in 22m20s
CI / coverage (pull_request) Successful in 1h5m5s
Robot Framework E2E test suite exercising the complete M5 success
criteria verification sequence:

- Large project creation with 10,000+ simulated files
- Resource registration and linking to a project
- Indexing verification (resource linked, project shows correctly)
- Context tier management (hot/warm/cold) via ContextConfig
- ACMS v1 context policy set/show with persistence round-trip
- Phase view inheritance (default → strategize → execute → apply)
- Scoped context output per ACMS phase with narrowing size limits
- Policy clear and inheritance fallback
- ContextView validation rules (size limits, VALID_PHASES)

Nine subcommands in the Python helper, each printing a sentinel
string on success. All subcommands verified passing locally.

Closes #406
2026-02-26 16:32:01 +00:00