test/cli-docstring-example-validation
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
f0442e835d |
feat(acms): plan execution leverages ACMS context for LLM calls
CI / typecheck (pull_request) Successful in 50s
CI / lint (pull_request) Successful in 3m31s
CI / security (pull_request) Successful in 4m14s
CI / build (pull_request) Successful in 26s
CI / quality (pull_request) Successful in 3m45s
CI / helm (pull_request) Successful in 40s
CI / integration_tests (pull_request) Successful in 3m57s
CI / unit_tests (pull_request) Successful in 4m24s
CI / docker (pull_request) Successful in 1m29s
CI / coverage (pull_request) Successful in 10m54s
CI / benchmark-publish (pull_request) Has been skipped
CI / e2e_tests (pull_request) Successful in 15m35s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-regression (pull_request) Successful in 1h0m6s
Integrate ACMS execute-phase context assembly into LLMExecuteActor and inject assembled context into execute prompts with resilient fallback when assembly fails or returns empty output. Wire the plan CLI executor to use an ACMS-backed execute context assembler, add Behave coverage for context injection/fallback/empty context, and extend Robot M5 verification helpers to assert execute-phase ACMS context usage. ISSUES CLOSED: #850 |
||
|
|
3eecb79003 |
test(plan): TDD failing tests for checkpoint real rollback (bug #822)
CI / lint (pull_request) Successful in 16s
CI / quality (pull_request) Successful in 46s
CI / typecheck (pull_request) Successful in 56s
CI / security (pull_request) Successful in 56s
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 29s
CI / e2e_tests (pull_request) Successful in 2m7s
CI / unit_tests (pull_request) Successful in 3m8s
CI / docker (pull_request) Successful in 9s
CI / integration_tests (pull_request) Successful in 3m57s
CI / coverage (pull_request) Successful in 7m48s
CI / benchmark-regression (pull_request) Successful in 40m48s
TDD expected-fail tests proving bug #822 exists: CheckpointService.rollback_to_checkpoint() returns a successful RollbackResult but does not execute git reset --hard. Files modified after the checkpoint remain unchanged after rollback. Also fixes Robot Framework timeout robustness across the entire test suite: all Run Process calls now use on_timeout=kill (prevents SIGTERM-induced -15 exit codes under CI load) and timeouts increased to 120s (prevents premature kills during heavy parallel execution). ISSUES CLOSED: #839 |
||
|
|
eb770643c2
|
test(e2e): validate M5 acceptance criteria for v3.4.0 milestone closure
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 16s
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 19s
CI / e2e_tests (pull_request) Successful in 32s
CI / security (pull_request) Successful in 37s
CI / typecheck (pull_request) Successful in 39s
CI / unit_tests (pull_request) Successful in 3m1s
CI / integration_tests (pull_request) Successful in 3m39s
CI / docker (pull_request) Successful in 40s
CI / coverage (pull_request) Successful in 6m49s
CI / lint (push) Successful in 12s
CI / build (push) Successful in 15s
CI / quality (push) Successful in 17s
CI / e2e_tests (push) Successful in 30s
CI / typecheck (push) Successful in 36s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 49s
CI / unit_tests (push) Successful in 3m16s
CI / integration_tests (push) Successful in 3m37s
CI / docker (push) Successful in 51s
CI / coverage (push) Successful in 6m40s
CI / benchmark-publish (push) Successful in 20m12s
CI / benchmark-regression (pull_request) Successful in 37m11s
Add four CLI-based integration test cases to the M5 E2E verification suite that exercise the exact commands from the v3.4.0 milestone description via real subprocess calls to `python -m cleveragents`. CLI test cases: - CLI Project Create Large Project: `agents project create local/large-project` - CLI Resource Add Git Checkout: `agents resource add git-checkout ...` - CLI Project Link Resource: `agents project link-resource ...` - CLI Project Show Displays Linked Resource: `agents project show ...` Each test creates an isolated temp directory with its own CLEVERAGENTS_HOME, initialises a workspace via `agents init`, runs the target CLI command as a subprocess with `on_timeout=kill`, asserts zero exit code and verifies expected output, then tears down the temp directory. Bug fix: ProjectResourceLinkRepository.create_link() and remove_link() only called session.flush() without session.commit(), causing linked resource data to be silently lost between sessions. Added session.commit() to both methods, plus finally: session.close() to match the session-factory lifecycle pattern used by all other mutating repository methods. Added session.refresh() and session.expunge() before return in create_link() so the returned ORM instance is fully loaded and usable in detached state after session close. Regression guard: Added two cross-session persistence Behave scenarios (project_repository.feature) that open a new session from the same engine after create_link/remove_link and verify the operation was durably committed. Test improvements from review feedback (rounds 1 and 2): - Per-test CLEVERAGENTS_HOME isolation via env: override on all Run Process calls to prevent shared state between tests. - Stronger ULID-based assertions: CLI tests capture the resource_id ULID from resource show output and verify the exact ID in project show output. - Regex-based branch assertion (branch.*main) instead of generic string match. - Test 4 differentiated from Test 3 by verifying resource show independently after linking and asserting the specific resource ULID. - Documentation noting context tier and ACMS criteria are validated at the Python API level via helper_m5_e2e_verification.py. - Updated ProjectResourceLinkRepository class docstring documenting commit and close behaviour of mutating methods. - CHANGELOG entry for both test additions and production bug fix. - Redundant Library imports removed, --format plain on project list, comments explaining fake repo directories, standardised Run Process line style. Context/ACMS CLI coverage is intentionally not added because the CLI does not yet expose dedicated context/ACMS inspection commands. These criteria are validated at the Python API level via helper_m5_e2e_verification.py. ISSUES CLOSED: #496 |
||
|
|
fd1612c38f |
test(e2e): verify M5 success criteria — ACMS and large-project context
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 17s
CI / lint (pull_request) Successful in 21s
CI / quality (pull_request) Successful in 20s
CI / security (pull_request) Successful in 54s
CI / typecheck (pull_request) Successful in 56s
CI / integration_tests (pull_request) Successful in 4m21s
CI / unit_tests (pull_request) Successful in 10m21s
CI / docker (pull_request) Successful in 39s
CI / benchmark-regression (pull_request) Successful in 22m20s
CI / coverage (pull_request) Successful in 1h5m5s
Robot Framework E2E test suite exercising the complete M5 success criteria verification sequence: - Large project creation with 10,000+ simulated files - Resource registration and linking to a project - Indexing verification (resource linked, project shows correctly) - Context tier management (hot/warm/cold) via ContextConfig - ACMS v1 context policy set/show with persistence round-trip - Phase view inheritance (default → strategize → execute → apply) - Scoped context output per ACMS phase with narrowing size limits - Policy clear and inheritance fallback - ContextView validation rules (size limits, VALID_PHASES) Nine subcommands in the Python helper, each printing a sentinel string on success. All subcommands verified passing locally. Closes #406 |