7faab96582c813541b0f7806b96142e9ef654c6e
43 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
4cca36dcd4 |
test(auto-inf-3): Consolidate Behave database fixtures via shared factory
CI / lint (pull_request) Successful in 58s
CI / typecheck (pull_request) Successful in 1m0s
CI / quality (pull_request) Successful in 48s
CI / security (pull_request) Successful in 1m8s
CI / helm (pull_request) Successful in 42s
CI / build (pull_request) Successful in 44s
CI / push-validation (pull_request) Successful in 26s
CI / unit_tests (pull_request) Successful in 4m31s
CI / docker (pull_request) Successful in 1m32s
CI / integration_tests (pull_request) Successful in 8m5s
CI / coverage (pull_request) Successful in 8m54s
CI / status-check (pull_request) Successful in 3s
Extended features/mocks/test_uow_factory.py with: - use_test_uow(context) function to attach test UoW to Behave context with automatic cleanup - cleanup_test_uow(context) function for teardown - Comprehensive docstrings and examples Updated features/environment.py: - Added cleanup_test_uow() call in after_scenario hook ISSUES CLOSED: #9541 |
||
|
|
d25a060c58 |
Revert "feat(ci): implement TDD bug tag quality gate for bug fix PRs"
CI / lint (pull_request) Successful in 50s
CI / quality (pull_request) Successful in 1m2s
CI / push-validation (pull_request) Successful in 39s
CI / helm (pull_request) Successful in 43s
CI / typecheck (pull_request) Successful in 1m31s
CI / security (pull_request) Successful in 1m32s
CI / build (pull_request) Successful in 1m3s
CI / integration_tests (pull_request) Successful in 3m42s
CI / unit_tests (pull_request) Successful in 6m13s
CI / docker (pull_request) Successful in 2m2s
CI / coverage (pull_request) Successful in 11m15s
CI / status-check (pull_request) Successful in 3s
CI / build (push) Successful in 53s
CI / lint (push) Successful in 1m1s
CI / helm (push) Successful in 26s
CI / quality (push) Successful in 1m16s
CI / typecheck (push) Successful in 1m20s
CI / security (push) Successful in 1m37s
CI / benchmark-regression (push) Failing after 40s
CI / push-validation (push) Successful in 21s
CI / integration_tests (push) Successful in 3m17s
CI / unit_tests (push) Successful in 4m28s
CI / e2e_tests (push) Successful in 3m22s
CI / docker (push) Successful in 1m52s
CI / coverage (push) Successful in 12m50s
CI / status-check (push) Successful in 3s
CI / benchmark-publish (push) Successful in 1h20m0s
This reverts commit
|
||
|
|
e8d2f76466
|
feat(ci): implement TDD bug tag quality gate for bug fix PRs
CI / push-validation (pull_request) Successful in 45s
CI / helm (pull_request) Successful in 57s
CI / build (pull_request) Successful in 1m8s
CI / lint (pull_request) Successful in 1m31s
CI / tdd_quality_gate (pull_request) Failing after 1m26s
CI / quality (pull_request) Successful in 1m30s
CI / typecheck (pull_request) Successful in 1m43s
CI / security (pull_request) Successful in 1m42s
CI / e2e_tests (pull_request) Failing after 4m10s
CI / integration_tests (pull_request) Successful in 5m2s
CI / unit_tests (pull_request) Successful in 6m6s
CI / docker (pull_request) Successful in 1m32s
CI / coverage (pull_request) Successful in 12m12s
CI / status-check (push) Blocked by required conditions
CI / status-check (pull_request) Failing after 3s
CI / tdd_quality_gate (push) Has been skipped
CI / benchmark-regression (push) Failing after 1m8s
CI / build (push) Successful in 1m2s
CI / lint (push) Successful in 1m16s
CI / helm (push) Successful in 45s
CI / push-validation (push) Successful in 45s
CI / quality (push) Successful in 1m37s
CI / typecheck (push) Successful in 2m1s
CI / security (push) Successful in 2m1s
CI / integration_tests (push) Successful in 3m34s
CI / unit_tests (push) Successful in 5m14s
CI / docker (push) Successful in 1m37s
CI / coverage (push) Failing after 19m17s
CI / benchmark-publish (push) Successful in 1h20m43s
CI / e2e_tests (push) Successful in 4m13s
Add an automated quality gate that enforces TDD bug fix workflow rules on pull requests. The gate parses PR descriptions for bug-closing keywords (Closes/Fixes/Resolves #N, ISSUES CLOSED: #N), searches the codebase for corresponding TDD tests tagged @tdd_bug_N, and verifies that @tdd_expected_fail tags have been removed. Key components: - scripts/tdd_quality_gate.py: Main quality gate script with PR description parsing, TDD test discovery, and tag removal verification. All public functions validate arguments fail-fast and are statically typed. - noxfile.py: New tdd_quality_gate session that reads PR_DESCRIPTION from the environment and runs the quality gate script. - .forgejo/workflows/ci.yml: New tdd_quality_gate CI job that runs only on pull_request events, passing the PR body as PR_DESCRIPTION. - features/tdd_quality_gate.feature: 46 Behave scenarios covering PR parsing, TDD test search, tag removal verification, full gate logic, robot diff handling, edge cases, argument validation, bool guards, co-located bug false-positive guard, and main() CLI entry point. - features/steps/tdd_quality_gate_steps.py: Step definitions for all Behave scenarios using temporary directories for isolation. - robot/tdd_quality_gate.robot: 15 Robot Framework integration tests exercising the gate end-to-end via a helper subprocess. - robot/helper_tdd_quality_gate.py: Helper script for Robot tests with sentinel-based sub-commands. Review-round fixes applied: - check_expected_fail_removed now uses _contains_tag_token for word-boundary matching (avoids false positives on partial tag names) - Diff expected-fail removal detection tracks flags at file level instead of per-hunk (fixes false negatives when tags span hunks) - parse_bug_refs filters out issue number zero - Redundant double error reporting eliminated (file-level check short-circuits the diff-level check) - run_quality_gate returns (errors, bug_refs) tuple to avoid redundant re-parsing in main() - Regex compilation cached via functools.lru_cache - Nox session no longer installs the full project (stdlib only) - CI checkout uses fetch-depth: 0 for reliable merge-base resolution Review-round 2 fixes applied: - _diff_has_expected_fail_removal_for_bug now requires the removed line to contain both the expected-fail tag and the specific bug tag (fixes false positives when two bugs share the same test file) - check_expected_fail_removed error messages use the correct tag prefix per file type (@tdd_bug_N for .feature, tdd_bug_N for .robot) - bool values rejected by bug-number validation guards in find_tdd_tests, check_expected_fail_removed, and _diff_has_expected_fail_removal_for_bug - File-read error handling catches UnicodeDecodeError alongside OSError (root-safe unreadable-file handling via invalid-UTF-8 test fixture) - Temp directory cleanup added to after_scenario hook in environment.py - 8 new Behave scenarios: bool type guards (2), co-located bug false-positive regression (1), run_quality_gate argument validation (3), and main() CLI entry point exit codes (2) Review-round 3 fixes applied: - Synthetic PR diff helper (_default_pr_diff_for_bug_refs) now auto-detects .robot vs .feature file type from the temp search tree and generates the matching diff format (fixes under-tested robot-format diff code path in multi-bug integration scenarios) - check_expected_fail_removed test step now filters files by bug tag via find_tdd_tests before checking (matches production path in run_quality_gate) - after_scenario temp directory cleanup no longer sets context.temp_dir = None (fixes cleanup conflict with cli_init_yes_flag_steps.py cleanup functions that run after hooks) - 2 new Behave scenarios: multi-line PR description parsing, and non-string pr_diff type guard for run_quality_gate ISSUES CLOSED: #629 |
||
|
|
ac84f314ff | test(behave): Fix concurrent BDD tests failures | ||
|
|
b846ab5cd7 |
fix(acms): rebase ACMS index data model onto master to fix unit test failures
Rebases the ACMS index data model and file traversal engine implementation onto the current master branch to resolve merge conflicts and fix unit test failures caused by the PR branch being 321 commits behind master. All ACMS-specific changes are preserved: - src/cleveragents/acms/index.py: ACMS index data model with hot/warm/cold/archive tiers - src/cleveragents/acms/__init__.py: Updated exports - features/acms/index_data_model_and_traversal.feature: BDD feature file - features/steps/acms_index_data_model_traversal_steps.py: Step definitions - features/environment.py: Added temp_dir cleanup hook - CHANGELOG.md: ACMS entry preserved ISSUES CLOSED: #9579 |
||
|
|
7164b04019 |
refactor(providers): unify provider factory behind single source of truth
CI / lint (push) Successful in 1m1s
CI / build (push) Successful in 53s
CI / quality (push) Successful in 1m23s
CI / typecheck (push) Successful in 1m24s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 1m42s
CI / push-validation (push) Successful in 21s
CI / helm (push) Successful in 25s
CI / e2e_tests (push) Successful in 4m6s
CI / integration_tests (push) Successful in 4m12s
CI / unit_tests (push) Successful in 6m39s
CI / docker (push) Successful in 1m44s
CI / coverage (push) Successful in 10m51s
CI / status-check (push) Successful in 2s
CI / benchmark-publish (push) Successful in 1h17m10s
CI / benchmark-publish (pull_request) Has been skipped
CI / helm (pull_request) Successful in 33s
CI / build (pull_request) Successful in 50s
CI / lint (pull_request) Successful in 1m7s
CI / quality (pull_request) Successful in 1m9s
CI / typecheck (pull_request) Successful in 1m20s
CI / security (pull_request) Successful in 1m25s
CI / push-validation (pull_request) Successful in 19s
CI / integration_tests (pull_request) Successful in 3m20s
CI / e2e_tests (pull_request) Failing after 5m3s
CI / unit_tests (pull_request) Successful in 6m40s
CI / docker (pull_request) Successful in 1m30s
CI / coverage (pull_request) Successful in 11m9s
CI / status-check (pull_request) Failing after 3s
CI / benchmark-regression (pull_request) Failing after 1m11s
Extract _get_api_key() and _create_provider_instance() as a single internal factory that handles all provider types, including MOCK. Both create_llm() and create_ai_provider() now delegate to _create_provider_instance(), eliminating the duplicated if/elif provider-switching chains and the divergence between the two code paths. Adding a new provider now requires touching exactly one place. Key changes: - New _get_api_key(provider_type) consolidates API key lookup and validation that was previously duplicated across create_llm(), _create_provider_llm(), and every branch of create_ai_provider(). - New _create_provider_instance() is the unified raw-LLM factory, replacing _create_provider_llm() and the specialised adapter constructors in create_ai_provider(). MOCK is now handled here via FakeListLLM. - create_ai_provider() wraps all providers uniformly in LangChainChatProvider instead of using specialised adapters (OpenAIChatProvider, AnthropicChatProvider, GoogleChatProvider, OpenRouterChatProvider) for the four primary providers. - create_llm() drops its own API key validation block; validation now happens inside _create_provider_instance() via _get_api_key(). - All BDD scenarios updated: _create_provider_llm references become _create_provider_instance, and provider-type assertions on create_ai_provider results are updated to LangChainChatProvider. Review fixes (Cycle 2): - Add _coerce_env_bool() for consistent boolean env-var coercion in CLEVERAGENTS_ALLOW_MOCK_PROVIDER checks. - Introduce _ApiKeyMissing sentinel to distinguish 'not provided' from None in provider configuration. - Set MOCK supports_streaming=False in DEFAULT_CAPABILITIES to align with FakeListLLM semantics. - Make MOCK is_configured conditional on CLEVERAGENTS_ALLOW_MOCK_PROVIDER. - Use FakeListLLM(responses=['mock response'] * 10) to prevent IndexError in multi-step workflows. - Strip max_retries from kwargs before passing to LangChain constructors. - Extract _resolve_provider_type() helper shared by create_llm() and create_ai_provider() to reduce duplication and improve readability. - Ensure __api_key_sentinel flows through factory_kwargs even when api_key is None, avoiding double _get_api_key() calls. - Add CHANGELOG env-var documentation and providers.md env-var table. - Resolve template DB lock issues by persisting in-memory SQLite via VACUUM INTO instead of direct file creation on tmpfs. ISSUES CLOSED: #10949 |
||
|
|
8313096b47 |
fix(tests): patch _get_session_service directly to eliminate parallel-worker race condition
CI / benchmark-publish (push) Failing after 43s
CI / lint (push) Successful in 52s
CI / quality (push) Successful in 1m0s
CI / typecheck (push) Successful in 1m21s
CI / security (push) Successful in 1m33s
CI / helm (push) Successful in 26s
CI / push-validation (push) Successful in 25s
CI / build (push) Successful in 37s
CI / integration_tests (push) Successful in 3m46s
CI / unit_tests (push) Successful in 4m31s
CI / e2e_tests (push) Successful in 3m47s
CI / docker (push) Successful in 1m32s
CI / coverage (push) Successful in 11m25s
CI / status-check (push) Successful in 5s
CI / benchmark-publish (pull_request) Has been skipped
CI / coverage (pull_request) Successful in 11m39s
CI / build (pull_request) Successful in 1m17s
CI / unit_tests (pull_request) Successful in 6m34s
CI / e2e_tests (pull_request) Successful in 4m47s
CI / status-check (pull_request) Waiting to run
CI / push-validation (pull_request) Successful in 29s
CI / typecheck (pull_request) Successful in 1m55s
CI / quality (pull_request) Successful in 1m56s
CI / integration_tests (pull_request) Successful in 5m36s
CI / helm (pull_request) Successful in 34s
CI / lint (pull_request) Successful in 1m29s
CI / security (pull_request) Successful in 1m56s
CI / docker (pull_request) Successful in 1m35s
The root cause was a race condition in parallel Behave workers caused by mutating a module-level singleton (_service) in cleveragents.cli.commands.session. Concurrent cleanup in one worker could reset _service to None while another worker was still using it, leading to intermittent tell command test failures and exit code 1. The fix patches the _get_session_service function directly in the affected test steps (session_cli_coverage_boost_steps.py and session_cli_uncovered_branches_steps.py) to avoid mutating the module-level _service, and adds a reset call in features/environment.py's after_scenario to ensure the singleton is cleared between scenarios, preventing stale service instances from leaking across tests. Closes #9121 |
||
|
|
9d21ba5b64 | style(test): rename _orig_time_sleep/_orig_asyncio_sleep to _original_time_sleep/_original_asyncio_sleep for consistency | ||
|
|
5812e0599a |
[AUTO-INF-3B] features/environment.py uses # type: ignore comments in _install_fast_sleep_patch() violating CONTRIBUTING.md strict typing policy
Removed 6 `# type: ignore` suppressions from `_install_fast_sleep_patch()` in `features/environment.py`. Used `cast(Any, module)` + direct attribute assignment instead of `setattr()` (which ruff B010 disallows) and `# type: ignore` comments. Added `from collections.abc import Callable` and `from typing import cast` imports. Added new feature file `features/test_infra_sleep_patch.feature` with 4 scenarios verifying the behavior. Added new steps file `features/steps/test_infra_sleep_patch_steps.py`. ISSUES CLOSED: #9993 |
||
|
|
f67e8a2e07
|
fix(testing): document and harden non-AssertionError guard in apply_tdd_inversion to reduce flaky CI
CI / lint (pull_request) Successful in 18s
CI / build (pull_request) Successful in 17s
CI / helm (pull_request) Successful in 18s
CI / quality (pull_request) Successful in 53s
CI / typecheck (pull_request) Successful in 56s
CI / security (pull_request) Successful in 57s
CI / push-validation (pull_request) Successful in 40s
CI / unit_tests (pull_request) Successful in 3m13s
CI / integration_tests (pull_request) Successful in 4m23s
CI / e2e_tests (pull_request) Successful in 4m37s
CI / docker (pull_request) Successful in 1m34s
CI / coverage (pull_request) Successful in 10m46s
CI / status-check (pull_request) Successful in 1s
CI / lint (push) Successful in 17s
CI / quality (push) Successful in 17s
CI / build (push) Successful in 24s
CI / helm (push) Successful in 24s
CI / push-validation (push) Successful in 37s
CI / typecheck (push) Successful in 52s
CI / security (push) Successful in 52s
CI / e2e_tests (push) Successful in 3m13s
CI / unit_tests (push) Successful in 6m37s
CI / integration_tests (push) Successful in 6m39s
CI / docker (push) Successful in 1m35s
CI / coverage (push) Successful in 11m13s
CI / status-check (push) Successful in 1s
Surface the non-AssertionError guard warning in standard Behave output by emitting to stderr in addition to the structured logger, and add infrastructure coverage that asserts this guard path is visible during test runs. Document the @tdd_expected_fail expectation that bug-signaling failures must use AssertionError so infrastructure exceptions are not accidentally treated as expected bug failures. ISSUES CLOSED: #8294 |
||
|
|
5437c73420 |
perf(test): replace per-instance _database_initialized flag with process-global cache (#1264)
CI / lint (push) Has been cancelled
CI / e2e_tests (push) Has been cancelled
CI / typecheck (push) Has been cancelled
CI / build (push) Has been cancelled
CI / integration_tests (push) Has been cancelled
CI / quality (push) Has been cancelled
CI / benchmark-publish (push) Has been cancelled
CI / security (push) Has been cancelled
CI / helm (push) Has been cancelled
CI / unit_tests (push) Has been cancelled
CI / docker (push) Has been cancelled
CI / status-check (push) Has been cancelled
CI / coverage (push) Has been cancelled
CI / benchmark-regression (push) Has been cancelled
Add a process-global `_INITIALIZED_DBS: set[str]` to `features/environment.py` that caches database URLs after `_fast_init_or_upgrade` has processed them. On subsequent calls with the same URL, the function returns immediately — skipping all URL parsing, path extraction, prefix matching, and `stat()` syscalls that previously executed on every new `UnitOfWork` instance. The cache is cleared at the start of each scenario in `before_scenario` to prevent cross-scenario state leaks, since each scenario receives fresh temp-DB paths via `tempfile.mktemp()`. Four new Behave scenarios validate cache hits on repeated calls, cache clearing between scenarios, and non-caching of non-matching-prefix URLs. Estimated savings: ~65,000 unnecessary function-body executions eliminated per full test run (~7 UnitOfWork instantiations × ~10,700 scenarios). ISSUES CLOSED: #735 Reviewed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com> Co-authored-by: Brent E. Edwards <brent.edwards@cleverthis.com> Co-committed-by: Brent E. Edwards <brent.edwards@cleverthis.com> |
||
|
|
1878998b7a |
refactor(testing): rename tdd_bug/tdd_bug_N tags to tdd_issue/tdd_issue_N
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 25s
CI / lint (pull_request) Successful in 3m19s
CI / typecheck (pull_request) Successful in 3m56s
CI / security (pull_request) Successful in 4m6s
CI / quality (pull_request) Successful in 4m10s
CI / integration_tests (pull_request) Successful in 7m20s
CI / unit_tests (pull_request) Successful in 7m44s
CI / docker (pull_request) Successful in 1m8s
CI / e2e_tests (pull_request) Successful in 13m1s
CI / coverage (pull_request) Successful in 12m25s
CI / status-check (pull_request) Successful in 1s
CI / build (push) Successful in 26s
CI / lint (push) Successful in 3m31s
CI / quality (push) Successful in 3m40s
CI / typecheck (push) Successful in 3m57s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 4m1s
CI / integration_tests (push) Successful in 8m55s
CI / unit_tests (push) Successful in 9m14s
CI / docker (push) Successful in 9s
CI / e2e_tests (push) Successful in 11m42s
CI / coverage (push) Successful in 11m37s
CI / status-check (push) Successful in 4s
CI / benchmark-publish (push) Successful in 29m25s
CI / benchmark-regression (pull_request) Successful in 54m8s
Rename the TDD tag system from tdd_bug/tdd_bug_<N> to tdd_issue/tdd_issue_<N> across the entire codebase. The tdd_expected_fail tag is unchanged. The TDD expected-failure workflow is not limited to bug fixes — it applies equally to any issue type (features, tasks, refactors). The _bug suffix was misleading and narrowed the perceived scope. The new _issue suffix accurately reflects that the TDD tagging system applies to any Forgejo issue. Changes span 92 files: - features/environment.py: validate_tdd_tags(), should_invert_result(), and apply_tdd_inversion() updated — regex, variables, error messages - robot/tdd_expected_fail_listener.py: _validate_tdd_tags(), _should_invert_result(), start_test(), end_test() updated consistently - 33 Behave .feature files: all @tdd_bug/@tdd_bug_<N> tags renamed - 29 Robot .robot files: all tdd_bug/tdd_bug_<N> tags renamed - 3 Robot fixture files renamed (tdd_bug_alone, tdd_missing_tdd_bug, tdd_expected_fail_missing_bug_n) with content and references updated - Tag validation tests and helpers updated (function names, command dispatch keys, output strings, fixture references) - CONTRIBUTING.md: section renamed from 'TDD Bug Test Tags' to 'TDD Issue Test Tags', all tag references and examples updated - noxfile.py: comment references updated - Step definition files, mock helpers, and benchmark files: docstring references updated ISSUES CLOSED: #965 |
||
|
|
a2113deace |
fix(a2a): suppress stdout/stderr in facade bootstrap to prevent test pollution
CI / benchmark-publish (pull_request) Has been skipped
CI / build (pull_request) Successful in 41s
CI / lint (pull_request) Successful in 3m19s
CI / quality (pull_request) Successful in 3m44s
CI / typecheck (pull_request) Successful in 3m56s
CI / security (pull_request) Successful in 4m5s
CI / e2e_tests (pull_request) Successful in 8m33s
CI / integration_tests (pull_request) Successful in 8m37s
CI / unit_tests (pull_request) Successful in 8m42s
CI / docker (pull_request) Successful in 1m12s
CI / coverage (pull_request) Successful in 10m58s
CI / status-check (pull_request) Successful in 1s
CI / lint (push) Successful in 3m28s
CI / build (push) Successful in 14s
CI / typecheck (push) Successful in 3m52s
CI / benchmark-regression (push) Has been skipped
CI / security (push) Successful in 4m0s
CI / quality (push) Successful in 4m5s
CI / integration_tests (push) Successful in 6m56s
CI / unit_tests (push) Successful in 7m15s
CI / docker (push) Successful in 1m12s
CI / e2e_tests (push) Successful in 8m36s
CI / coverage (push) Successful in 11m35s
CI / status-check (push) Successful in 2s
CI / benchmark-publish (push) Successful in 28m44s
CI / benchmark-regression (pull_request) Successful in 51m20s
The _notify_facade and _facade_dispatch functions call get_container() during lazy facade construction. This can trigger structlog output that corrupts CLI stdout captured by CliRunner in tests. Wrap facade construction in redirect_stdout/redirect_stderr to suppress any side-effect output. Also reset the facade singleton in after_scenario for test isolation. |
||
|
|
c8cd7eab82
|
feat(testing): implement @tdd_expected_fail tag handling in Behave environment
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 14s
CI / build (pull_request) Successful in 16s
CI / quality (pull_request) Successful in 17s
CI / security (pull_request) Successful in 40s
CI / typecheck (pull_request) Successful in 55s
CI / unit_tests (pull_request) Successful in 3m0s
CI / integration_tests (pull_request) Successful in 3m26s
CI / docker (pull_request) Successful in 44s
CI / coverage (pull_request) Successful in 5m28s
CI / lint (push) Successful in 13s
CI / build (push) Successful in 14s
CI / quality (push) Successful in 16s
CI / security (push) Successful in 36s
CI / typecheck (push) Successful in 52s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 3m17s
CI / integration_tests (push) Successful in 3m29s
CI / docker (push) Successful in 42s
CI / coverage (push) Successful in 6m47s
CI / benchmark-publish (push) Successful in 18m40s
CI / benchmark-regression (pull_request) Successful in 37m4s
Implemented the three-tag TDD bug-capture system in Behave environment hooks: - Added tag validation in before_scenario: @tdd_bug_<N> requires @tdd_bug, @tdd_expected_fail requires both @tdd_bug and @tdd_bug_<N> - Added result inversion via Scenario.run() wrapper installed in before_all: scenarios tagged @tdd_expected_fail that fail are reported as passed (expected failure), and scenarios that unexpectedly pass are reported as failed with guidance to remove the tag - Added helper functions validate_tdd_tags() and should_invert_result() - Added inline documentation referencing CONTRIBUTING.md > TDD Bug Test Tags - Added Behave test scenarios for tag validation and inversion behavior - Extract apply_tdd_inversion() as a public, testable function that encapsulates all inversion logic with proper guards - Refactored handle_tdd_expected_fail() to delegate to apply_tdd_inversion() after tag validation, eliminating ~55 lines of duplicated inversion logic - Tag validation errors in handle_tdd_expected_fail() are now logged at WARNING level instead of being silently swallowed - Add hook_failed guard: never invert infrastructure/hook errors - Add was_dry_run guard: skip inversion when no test actually ran - Add non-AssertionError guard: warn and skip inversion for exceptions that likely indicate infrastructure problems, not the captured bug - Log exception details at DEBUG level before clearing (previously discarded silently) - Attach synthetic AssertionError to last step on unexpected pass so the failure reason appears in standard Behave formatter output ISSUES CLOSED: #627 |
||
|
|
1ec525599b
|
fix(test): skip redundant Alembic migration check for existing test databases
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 14s
CI / build (pull_request) Successful in 16s
CI / quality (pull_request) Successful in 19s
CI / security (pull_request) Successful in 36s
CI / typecheck (pull_request) Successful in 38s
CI / unit_tests (pull_request) Successful in 2m59s
CI / docker (pull_request) Successful in 10s
CI / integration_tests (pull_request) Successful in 3m28s
CI / coverage (pull_request) Successful in 6m35s
CI / benchmark-regression (pull_request) Successful in 35m34s
CI / lint (push) Successful in 12s
CI / build (push) Successful in 14s
CI / quality (push) Successful in 16s
CI / security (push) Successful in 35s
CI / typecheck (push) Successful in 36s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 3m6s
CI / docker (push) Successful in 10s
CI / integration_tests (push) Successful in 3m26s
CI / coverage (push) Successful in 5m22s
CI / benchmark-publish (push) Successful in 19m46s
When a test database already exists and is non-empty (created by a prior step like `I have initialized a project`), _fast_init_or_upgrade in features/environment.py was falling through to _original_init_or_upgrade, running a full Alembic migration version check every time. This created redundant SQLAlchemy engines, caused SQLite lock contention under parallel execution, and triggered cumulative overhead from @database_retry decorators (wait_fixed(0.5) × 3 attempts on ~100 repository methods), leading to intermittent hangs and CI timeouts. The fix changes the existing-non-empty-database branch to return immediately instead of delegating to the original migration runner. This is safe because test databases are either template-copied (already at Alembic HEAD) or created by a prior step that already ran the full migration. The _SCENARIO_DB_PREFIXES guard ensures non-scenario databases (migration-runner unit tests) remain unaffected. Verified: 15/15 scenarios pass with both --processes 1 and --processes 16. ISSUES CLOSED: #726 |
||
|
|
aa5d5eeaf5 |
test(session): add TDD failing tests for session create DI error
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 16s
CI / quality (pull_request) Successful in 18s
CI / security (pull_request) Successful in 36s
CI / typecheck (pull_request) Successful in 38s
CI / unit_tests (pull_request) Successful in 2m42s
CI / integration_tests (pull_request) Successful in 3m13s
CI / docker (pull_request) Successful in 39s
CI / coverage (pull_request) Successful in 6m15s
CI / benchmark-regression (pull_request) Successful in 34m51s
Implement TDD bug-capture tests for bug #570 where `agents session create` fails because `_get_session_service()` calls `container.db()` which does not exist on the DI Container class (AttributeError). Same root cause as bug #554. Behave BDD scenarios tagged @tdd_bug @tdd_bug_570 @tdd_expected_fail exercise the real DI path (no mocks). Includes Robot Framework integration smoke tests with self-inverting helper and ASV benchmark baseline. ISSUES CLOSED: #631 |
||
|
|
06bbe48a9c |
test(session): add TDD failing tests for session list DI error
CI / lint (pull_request) Successful in 21s
CI / benchmark-publish (pull_request) Has been skipped
CI / quality (pull_request) Successful in 36s
CI / build (pull_request) Successful in 24s
CI / typecheck (pull_request) Successful in 59s
CI / security (pull_request) Successful in 1m14s
CI / unit_tests (pull_request) Successful in 3m27s
CI / docker (pull_request) Successful in 40s
CI / integration_tests (pull_request) Successful in 6m40s
CI / coverage (pull_request) Successful in 5m12s
CI / benchmark-regression (pull_request) Successful in 32m46s
Implement TDD bug-capture tests for bug #554 where `agents session list` fails because `_get_session_service()` calls `container.db()` which does not exist on the DI Container class (AttributeError). Behave BDD scenarios tagged @tdd_bug @tdd_bug_554 @tdd_expected_fail exercise the real DI path (no mocks) and assert correct behavior. The @tdd_expected_fail handler in environment.py inverts failed→passed while the bug is present, keeping CI green. Also adds: - @tdd_expected_fail infrastructure in features/environment.py (tag validation + status inversion in after_scenario hook) - behave-parallel exit logic fix to use summary-based failure detection (compatible with TDD status inversion) - Robot Framework integration smoke tests with self-inverting helper - ASV benchmark baseline for session list command throughput ISSUES CLOSED: #630 |
||
|
|
cf67ba0a86
|
feat(devcontainer): add lazy activation and lifecycle management
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 16s
CI / quality (pull_request) Successful in 18s
CI / security (pull_request) Successful in 35s
CI / typecheck (pull_request) Successful in 37s
CI / unit_tests (pull_request) Successful in 2m27s
CI / docker (pull_request) Successful in 49s
CI / integration_tests (pull_request) Successful in 3m46s
CI / coverage (pull_request) Successful in 4m56s
CI / lint (push) Successful in 12s
CI / build (push) Successful in 14s
CI / quality (push) Successful in 16s
CI / security (push) Successful in 32s
CI / typecheck (push) Successful in 36s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 4m7s
CI / integration_tests (push) Successful in 4m41s
CI / docker (push) Successful in 44s
CI / coverage (push) Successful in 4m56s
CI / benchmark-publish (push) Successful in 17m17s
CI / benchmark-regression (pull_request) Successful in 31m25s
Implemented lazy container activation for devcontainer-instance resources with ContainerLifecycleState enum tracking six states (inactive, starting, active, stopping, stopped, error) with validated transitions. Extended DevcontainerHandler with devcontainer up CLI integration and JSON output parsing for container start. Added periodic health checking via devcontainer exec ping with configurable interval. Added agents resource stop and agents resource rebuild CLI commands for manual lifecycle control. Wired session close and plan completion hooks to automatic container cleanup. Includes lifecycle state persistence in resource registry with timestamped transitions. Added Behave BDD tests, Robot integration tests, and ASV activation latency benchmarks. - Added remoteWorkspaceFolder absolute-path validation - Aligned spec: handler name, rebuild types, --yes flag on stop/rebuild - Added registry re-read in stop_container success path for consistency - Added session_id field to ContainerLifecycleTracker for scoped cleanup - Scoped stop_all_active_containers to session_id when provided - Wired _cleanup_devcontainers into fail_apply and fail_execute - Wired start_health_check into activate_container success path - Restructured facade session close to always run container cleanup even without session service (F4) - Re-read tracker from registry in activate_container success path - Added evict_terminal_trackers to cap registry growth - Updated devcontainer_resources.md: health check auto-start, scoped cleanup hooks, known limitations for eviction and sandbox_strategy - Wired evict_terminal_trackers into stop_all_active_containers so terminal-state trackers are actually evicted in production - Made stop_container idempotent: returns early when container is already in a terminal state instead of raising ValueError - Fixed benchmark health check thread leak in TimeActivationLatency by clearing registry after each timing loop - Added rebuild pass-through (--reset-container flag to devcontainer up) - Added host_workspace_path field on ContainerLifecycleTracker so health probes use the host-side path for devcontainer exec - Wired lazy activation into DevcontainerHandler.resolve() for devcontainer-instance resources in non-running states - Changed _default_strategy from SNAPSHOT to NONE (container itself provides isolation; SandboxFactory raises NotImplementedError for snapshot) - Restricted _STOPPABLE_TYPES to devcontainer-instance only (container-instance is not directly stoppable via CLI) ISSUES CLOSED: #514 |
||
|
|
febea8950f |
feat(acms): add ACMS v1 context pipeline
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 14s
CI / build (pull_request) Successful in 16s
CI / quality (pull_request) Successful in 18s
CI / security (pull_request) Successful in 32s
CI / typecheck (pull_request) Successful in 36s
CI / unit_tests (pull_request) Successful in 2m0s
CI / docker (pull_request) Successful in 51s
CI / integration_tests (pull_request) Successful in 3m5s
CI / coverage (pull_request) Successful in 4m20s
CI / benchmark-regression (pull_request) Successful in 28m14s
Implement the 10-component pluggable ACMS context assembly pipeline with three built-in strategies (relevance, recency, tiered), DI-based component injection, ULID-validated plan_id, largest-remainder budget allocation, and frozen Pydantic v2 domain models. Closes #188 |
||
|
|
4ad51561fc |
perf(tests): optimize medium-slow BDD features (10-100s tier)
Cap time.sleep and asyncio.sleep globally at 10ms in before_all to eliminate retry/backoff waits (tenacity wait_fixed, retry_auto_debug exponential backoff). Save originals as time._original_sleep and asyncio._original_sleep for tests that need real wall-clock delays (CircuitBreaker recovery, debounce timers, validation timeouts). Enhance template-DB patch: add "db." prefix to _SCENARIO_DB_PREFIXES and check db_path.stat().st_size > 0 so 0-byte auto-created SQLite files receive the template copy instead of falling through to real migrations. Replace subprocess.run CLI invocations with typer.testing.CliRunner in module_coverage, main_coverage_complete, and coverage_extras step files, eliminating ~6s Python cold-start overhead per call. Switch plan_persistence and action_persistence to in-memory SQLite by default; only cross-restart scenarios use file-based via an explicit Given step. Replace MigrationRunner.run_migrations with Base.metadata.create_all in plan_service_steps for freshly created in-memory engines. Removed leftover debug comment in environment.py. Total tier runtime: 565s -> 21s (96% reduction), 20 features all under 5s behave-internal time. ISSUES CLOSED: #480 |
||
|
|
0bbec4b77e |
perf(tests): optimize the 8 slowest BDD feature files
Optimized the 8 features accounting for 64% of total BDD test runtime (1,505s of 2,352s): - features/environment.py: Added _ensure_template_db() for direct behave invocations, expanded _SCENARIO_DB_PREFIXES to include "test_" for step-file-created DBs, added @mock_only tag support to skip unnecessary DB setup. - features/plan_commands_coverage.feature: Added @mock_only tag (fully mocked, no DB needed). - features/plan_service.feature: 14 actor-resolution scenarios now use lightweight in-memory plan service instead of heavyweight file-based DB + project init. - features/steps/plan_service_steps.py: New step_create_lightweight_plan_service for actor-resolution tests. - features/steps/services_coverage_steps.py: Extracted 3 helper functions to consolidate 8 near-identical Given steps (~200 lines of duplicated boilerplate removed). Per-feature results: services_coverage 245s->2.8s (99%), context_service 215s->2.1s (99%), project_service 140s->0.7s (99%), plan_service 215s->4.6s (98%), core_cli_commands 113s->4.2s (96%), cli_streaming 213s->6.4s (97%), plan_commands_coverage 116s->20s (83%), cli_plan_context_commands 248s->31.6s (87%). ISSUES CLOSED: #479 |
||
|
|
a8f7ed57cb |
perf(tests): reduce per-feature startup cost with shared fixtures and lazy imports
Created scripts/create_template_db.py that builds a pre-migrated SQLite template database using Base.metadata.create_all() + alembic stamp (~5ms for 34 tables, vs ~0.5-3s x 25 Alembic migrations per scenario). Nox unit_tests and coverage_report sessions generate the template before test execution and propagate CLEVERAGENTS_TEMPLATE_DB env var to all workers. features/environment.py before_all() installs a monkey-patch on MigrationRunner.init_or_upgrade that copies the template for fresh scenario temp DBs, falling through to real migrations for :memory:, existing files, and migration-runner unit tests. Quick wins: sleep(0.5) -> sleep(0.05) in cli_streaming wait step; removed redundant Background re-declaration in cli_streaming.feature scenario 7. ISSUES CLOSED: #483 |
||
|
|
64af753aab
|
feat(security): add permission system
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / build (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 19s
CI / security (pull_request) Successful in 32s
CI / typecheck (pull_request) Successful in 36s
CI / integration_tests (pull_request) Successful in 2m40s
CI / unit_tests (pull_request) Successful in 11m42s
CI / docker (pull_request) Successful in 40s
CI / benchmark-regression (pull_request) Successful in 21m35s
CI / coverage (pull_request) Successful in 45m10s
CI / lint (push) Successful in 12s
CI / quality (push) Successful in 18s
CI / build (push) Successful in 23s
CI / security (push) Successful in 32s
CI / typecheck (push) Successful in 57s
CI / benchmark-regression (push) Has been skipped
CI / integration_tests (push) Successful in 2m43s
CI / unit_tests (push) Successful in 12m13s
CI / benchmark-publish (push) Successful in 12m19s
CI / docker (push) Successful in 1m2s
CI / coverage (push) Successful in 1h26m6s
Implemented namespace/project/plan/skill permission model with role bindings (owner/admin/editor/viewer) and default deny policy. Added enforcement hooks at CLI/service boundaries that are server-only; local mode returns permissive defaults. Includes role enums, permission check service, role matrix documentation. Includes Behave BDD scenarios, Robot integration tests, ASV benchmarks, and reference documentation. ISSUES CLOSED: #344 |
||
|
|
c406781b86 |
fix(security): address PR #435 review feedback
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 17s
CI / build (pull_request) Successful in 20s
CI / typecheck (pull_request) Successful in 41s
CI / security (pull_request) Successful in 57s
CI / integration_tests (pull_request) Successful in 2m53s
CI / unit_tests (pull_request) Successful in 19m17s
CI / docker (pull_request) Successful in 1m1s
CI / benchmark-regression (pull_request) Successful in 26m11s
CI / coverage (pull_request) Successful in 47m20s
P0: reject register() after close_all() with RuntimeError. P1: catch CancelledError in close_all(), use WeakKeyDictionary for cancellation_reasons to prevent memory leak, guard StateManager update_state/reset/load_checkpoint/time_travel after close(). P2: contextlib.suppress in __del__ for partial construction, re-cancel pending tasks in cleanup_tasks_async, handle late tasks added during await window, guard AcpEventQueue.publish() after close with _is_closed flag and is_closed property, fix ASV TimeRegisterBatch crash. Tests: 5 new Behave scenarios (T1-T4 + is_closed), log handler and event loop cleanup in after_scenario (T5-T6). Docs: async_safety.md updated for register-after-close and state mutation guards. ISSUES CLOSED: #321 |
||
|
|
f2b9ccfde4 |
fix(test): isolate parallel behave subprocesses with per-scenario temp databases
Each behave-parallel worker previously resolved to the same file-based SQLite database (cleveragents.db or .cleveragents/db.sqlite) because before_scenario removed the CLEVERAGENTS_DATABASE_URL env var and the fallback paths are shared across processes. Under parallel execution this caused intermittent 'database is locked' and duplicate-project errors—most visibly in the coverage_report nox session. Replace the env-var removal in before_scenario with unique temp database paths via tempfile.mktemp(), giving every scenario its own isolated database file. Temp files are cleaned up in after_scenario. Also fix cli_streaming_steps.py where a Background + duplicate Given sequence triggered a duplicate project name collision, and update coverage_boost_steps.py so Settings-default assertions explicitly clear the env vars before testing pydantic defaults. |
||
|
|
79f91c51b1
|
Tests: ensuring tests have coverage and work. | ||
|
|
17fe46d925 |
test(behave): fix behave tests
CI / lint (pull_request) Successful in 18s
CI / typecheck (pull_request) Successful in 26s
CI / security (pull_request) Successful in 17s
CI / quality (pull_request) Successful in 14s
CI / build (pull_request) Successful in 12s
CI / behave (3.13) (pull_request) Failing after 3m50s
CI / docker (pull_request) Has been skipped
CI / helm (pull_request) Has been skipped
CI / coverage (pull_request) Failing after 4m20s
There had been over 100 behave tests failing. There should be none failing now. |
||
|
|
7ef5ebb695 |
feat: Add Q0: Pre-commit hooks setup.
This should automatically check for problems on build. |
||
|
|
6eda182440
|
chore: Removed all references to plandex including old code | ||
|
|
34a29d7cc8
|
Feat: Adding Actors support to database | ||
|
|
ea50a939cb
|
Added LangSmith tracing | ||
|
|
407b7ec53e
|
Feat: Created provider registry | ||
|
|
641f7c09d9
|
Fix: Added streaming to context commands | ||
|
|
496f565a57
|
Chore: Completed stage 1.5 | ||
|
|
568bcc3d76
|
Feat: Implemented basics of plan streaming and AutoDebugGraph | ||
|
|
6d2f26bef2
|
Feat: Added in plan and project commands and functionality | ||
|
|
d3fb2f9a74
|
feat: continued with phase 2 adding database migration scripts to the mix | ||
|
|
cbafe7fac4
|
Implemented unit of work | ||
|
|
4f05e8a016
|
feat(cli): Wrote core CLI commands and tests | ||
|
|
fb6f559780
|
feat(architecture): Added skeleton for phase 1 | ||
|
|
94c97d10f7
|
feat(discovery): Add implicit runtime behavior extraction and analysis | ||
|
|
b941ef9a1a
|
Significantly revamped the base project to use more robust static checking including benchmarks and integration test support | ||
|
|
f7a5d26c41
|
Modernized build tooling |