23 Commits

Author	SHA1	Message	Date
CoreRasurae	0c5b140d29	fix(database): include alembic files in package distribution ... Move alembic configuration and migration files from repository root into the Python package structure to ensure they are included in the wheel distribution. This fix resolves the FileNotFoundError when running `agents init` in Docker containers or any environment using the built wheel distribution. Changes: - Move alembic/ directory from repo root to src/cleveragents/infrastructure/database/migrations/ - Move alembic.ini to the same new location and update script_location setting - Update MigrationRunner._find_alembic_ini() to search from the new canonical location within the package - Update create_template_db.py to point to the new alembic.ini location - Update documentation references to reflect new migration file locations - Create __init__.py for migrations package - The env.py file is imported when running tests that verify all modules can be imported without errors. However, context.config is only available when alembic is actually running migrations, not during normal module imports. This caused an AttributeError when the test tried to import the migrations.env module. - Fix by using getattr() with a default value to safely access context.config, and guard all code that uses config with None checks. This allows the module to be safely imported while still functioning correctly during migrations. Testing: - Verified MigrationRunner can locate alembic.ini in new location - Tested agents init succeeds in creating project with database - Template database creation works correctly - All migration tests should pass without changes Alembic files now follow standard Python packaging conventions, making them automatically included in wheel distributions without special configuration. ISSUES CLOSED: #4180	2026-04-17 17:40:00 +00:00
HAL9000	449c33b752	style(tests): apply ruff format to run_behave_parallel.py ... Fix formatting of conditional expression in _is_btrfs_or_overlayfs() that was flagged by ruff format --check in CI.	2026-04-17 07:16:05 +00:00
Test	4fc4d7863c	fix(tests): prevent behave-parallel multiprocessing deadlock on btrfs/overlayfs ... Detect btrfs and overlayfs filesystems and automatically fall back to sequential mode to prevent deadlocks caused by SQLite WAL file locking and btrfs COW copy-up locks when multiple forked workers try to access the same files simultaneously. The fix adds a _is_btrfs_or_overlayfs() function that: 1. Attempts to detect the filesystem type using stat command 2. Falls back to reading /proc/mounts if stat fails 3. Returns True if the filesystem is btrfs or overlayfs The sequential mode condition is updated to include this check, ensuring that on affected filesystems, all features run in a single process instead of being split across multiple forked workers. ISSUES CLOSED: #9390	2026-04-17 07:16:05 +00:00
HAL9000	b8732dfc6f	fix(tests): resolve nox unit_tests timeout for agent_skills_loader and skill_search features ... Adjusted test running and file-detection logic to stabilize unit tests in overlayfs environments and improve target feature handling. - Modified scripts/run_behave_parallel.py to run sequentially when there are 2 or fewer feature files, avoiding fork deadlocks on overlayfs and reducing nox-based unit test timeouts for agent_skills_loader and skill_search features. - Updated noxfile.py to correctly detect feature files in posargs, fixing the prior logic that appended the "features/" directory when specific feature files were provided. This ensures precise test selection and avoids unnecessary path expansion. Rationale: These changes address the root causes of flaky unit test timeouts by preventing problematic forking behavior with small feature sets and by ensuring nox respects explicitly provided feature file paths. ISSUES CLOSED: #9374	2026-04-15 08:16:38 +00:00
hurui200320	8b2e0c81c5	fix(testing): print behave-parallel worker logs only for failed chunks ... In parallel mode, the behave runner previously replayed captured stdout/stderr for every worker chunk, creating noisy output that obscured failure diagnostics in CI and local runs. Changes to scripts/run_behave_parallel.py: - Added _chunk_has_failures() and _chunk_no_scenarios_ran() helpers to evaluate individual chunk summaries for failure/error/crash conditions. - Updated the aggregation loop in main() to conditionally replay captured stdout/stderr only for chunks whose summary indicates failures, errors, or no scenarios ran (crash detection). Passing chunks now suppress their output entirely. - Added robust exception handling in _worker_run_features() so that worker crashes produce a full traceback in stderr and return a crash summary with features.errors = 1, enabling the parent to detect the crash via _chunk_has_failures (and also _chunk_no_scenarios_ran, since no scenarios reached a terminal state) and replay the diagnostics. - The conditional replay uses summary-based checks rather than the raw runner.run() boolean, consistent with the existing exit-code logic. This avoids spurious log replay for @tdd_expected_fail scenarios whose runner.run() returns True even though the TDD inversion handler has corrected the scenario status to passed. - Existing summary merge, exit semantics, and the no-scenarios safety net are fully preserved. New Behave unit tests (17 scenarios) cover the chunk-level helpers, the conditional aggregation loop, the pure no-scenarios-ran path, stderr replay for non-crash failed chunks, and the worker crash path. New Robot integration tests (6 test cases) verify the same behavior end-to-end via the helper_behave_parallel_log_filtering.py script. Also updated: - CHANGELOG.md: add unreleased entry for this behavioral change. - features/steps/behave_parallel_log_filtering_steps.py: use contextlib.redirect_stdout/redirect_stderr instead of manual sys.stdout assignment; register module in sys.modules; document CWD requirement in _load_runner_module(). - robot/helper_behave_parallel_log_filtering.py: move import io to top-level; remove redundant inline imports; use contextlib for output capture; register module in sys.modules; document CWD requirement. Branch note: the canonical branch for this fix is bugfix/m3-behave-parallel-failed-chunk-logs. The PR head branch (bugfix/mX-behave-parallel-failed-chunk-logs) cannot be renamed via the Forgejo API; both branches are kept in sync at the same SHA. ISSUES CLOSED: #8351	2026-04-15 03:21:05 +00:00
HAL9000	4c0f3e1da9	fix(tests): fix create_template_db.py to create writable SQLite template database ... Added os.chmod(db_path, 0o664) after database creation to ensure the template database has writable permissions. This prevents sqlite3.OperationalError: attempt to write a readonly database when tests copy and modify the template during test setup. The template database is now created with rw-rw-r-- (664) permissions instead of the default rw-r--r-- (644), allowing the test runner process to write to it. ISSUES CLOSED: #9372	2026-04-14 22:10:07 +00:00
HAL9000	6559a0e9df	docs: integrate docs-writer automation tracking workflows ... - document docs-writer responsibilities and automation tracking requirements\n- enforce automation tracking label validation and clean coverage regression tags\n\nISSUES CLOSED: #7616 # Conflicts: # CHANGELOG.md # docs/development/automation-tracking.md # docs/development/docs-writer.md # mkdocs.yml	2026-04-14 14:11:47 +00:00
hurui200320	0bd8fbb296	fix(ci): resolve lint errors and remove stale tdd_expected_fail tag (#5264) ... ## Summary The `lint` and `integration_tests` CI jobs on `master` are currently failing. This PR fixes both: 1. **Lint (`nox -e lint`)** — 51 ruff violations in `scripts/validate_automation_tracking.py`: - Sorted imports per isort convention - Replaced deprecated `typing.List`/`Dict`/`Tuple`/`Optional` with builtin equivalents (`list`, `dict`, `tuple`) - Removed unused `Optional` import and unused local variable `issues` - Fixed trailing whitespace, blank-line whitespace, and 6 lines exceeding 88-char limit - Replaced `dict.keys()` with `dict` in membership tests - Added return type annotation to `main()` 2. **Integration tests (`nox -e integration_tests`)** — 1 failure in `robot/coverage_threshold.robot`: - Removed `tdd_expected_fail`, `tdd_issue`, and `tdd_issue_4305` tags from the `Noxfile Contains Coverage Threshold Constant` test - The bug this TDD tag tracked (issue #4305) is now fixed — `COVERAGE_THRESHOLD = 97` exists in `noxfile.py` - The TDD expected-fail listener was inverting the passing result to a failure ### Local verification - `nox -e lint` — All checks passed - `nox -e integration_tests` — 1962 tests, 1962 passed, 0 failed, 0 skipped ### Files changed | File | Change | |------|--------| | `scripts/validate_automation_tracking.py` | Fixed all 51 lint violations | | `robot/coverage_threshold.robot` | Removed stale `tdd_expected_fail` tag | Closes #5266 Reviewed-on: #5264 Co-authored-by: Rui Hu <rui.hu@cleverthis.com> Co-committed-by: Rui Hu <rui.hu@cleverthis.com>	2026-04-09 04:41:01 +00:00
freemo	f2fc8f771e	feat: add port checking to opencode-builder script ... - Add check_opencode_running() function to detect existing OpenCode server - Skip server startup if OpenCode already running on target port - Display prominent warning when connecting to existing server - Only manage server lifecycle when script starts its own server - Add OWN_SERVER flag to track server ownership - Update cleanup logic to leave existing servers running This enables multiple developers and automation scripts to safely run in parallel without conflicting server management.	2026-04-08 19:57:54 -04:00
freemo	a323f07783	feat: implement new automation tracking system for agent supervision ... Replace shared session state issue tracking with individual tracking issues per agent to reduce noise and improve searchability. **Agent Updates:** - session-persister: [AUTO-SESSION] prefix with cycle management - implementation-orchestrator: [AUTO-IMP-POOL] prefix for health reports - system-watchdog: [AUTO-WATCHDOG] prefix for system health - backlog-groomer: [AUTO-GROOMER] prefix + backup cleanup functionality - human-liaison: [AUTO-LIAISON] prefix for status updates **New Features:** - Standardized issue title format: [AUTO-<PREFIX>] <TYPE> (Cycle <N>) - Announcement format: [AUTO-<PREFIX>] Announce: <message> - Automatic cleanup to prevent issue accumulation - Required 'Automation Tracking' label for filtering - Validation script for format compliance **Documentation:** - Complete system documentation at docs/development/automation-tracking.md - Added to mkdocs.yml navigation - Validation script at scripts/validate_automation_tracking.py **Benefits:** - Reduced noise from shared tracking issue - Better searchability with agent-specific prefixes - Cleaner history per agent type - Easier debugging with focused issue threads - Automatic cleanup prevents accumulation Closes automation tracking system implementation requirements.	2026-04-08 21:28:46 +00:00
freemo	73afe58cfe	build: runner for opencode project-builder	2026-04-05 01:09:18 -04:00
freemo	02250473ad	fix(ci): restore all CI quality gates to passing on master ... Fix all failing CI quality gates (lint, unit_tests, format) without suppressing any quality enforcement. Root causes and fixes: 1. Format: features/steps/plan_namespaced_name_tdd_steps.py had trailing whitespace; fixed by running ruff format. 2. Unit tests - A2A JSON-RPC 2.0 migration (commit 9c6d6915) renamed A2aRequest fields (operation→method, request_id→id, a2a_version→jsonrpc) and A2aResponse fields (status+data→result, request_id→id) but did not update all step files and feature files: - a2a_jsonrpc_wire_format_steps.py: added use_step_matcher('re') and reset to 'parse' at end to prevent parallel test interference - a2a_facade_wiring_steps.py: updated operation= to method=, .status/.data to .result - a2a_facade_steps.py: updated request_id→id, a2a_version→jsonrpc, A2aResponse(request_id=..., status=...) to new API - m6_facade_steps.py: updated all old API usage - devcontainer_cleanup_steps.py: updated A2aRequest(operation=...) - plan_prompt_command_steps.py: updated A2aRequest(operation=...) - wf03_plan_prompt_confidence_steps.py: updated A2aRequest(operation=...) - consolidated_misc.feature: updated old A2aRequest/A2aResponse scenarios 3. Unit tests - Session CLI output changed (commit 0d5d9cf0 and others): - 'Session Created' → 'Session created' (lowercase) - 'Session Details' → 'Session Summary' - 'Sessions (N total)' → 'Sessions' - session list JSON: top-level 'total' → nested 'summary.total' - Fixed in: session_cli.feature, session_cli_coverage_boost.feature, session_cli_uncovered_branches.feature, session_list_error.feature, tdd_session_create_persist_steps.py 4. Unit tests - Plan list output changed (commit 1a07a891): - 'V3 Lifecycle Plans' → 'Plans' - 'Lifecycle Plans' → 'Plans' - Name column removed (restored in source) - Invariants column removed (restored in source) - Project truncation removed (restored in source) - Fixed in: plan_cli_cancel_revert_coverage.feature, plan_lifecycle_cli_coverage.feature, plan_cli_coverage_boost_steps.py, plan.py (source code restored) 5. Unit tests - Plan apply command now requires ULID (commit 300a5d6d): - plan_cli_coverage_r3.feature: updated 'PLAN-001' to valid ULID - plan_cli_coverage_r3_steps.py: added --yes flag, added new step for no-eligible-plans path 6. Unit tests - Various source code bugs: - ThoughtBlock: converted from @dataclass to Pydantic BaseModel (architecture test requires all dataclasses to use Pydantic) - session.py: added DatabaseError handling to export, import, tell commands - database.py: fixed rollback_to() to reuse checkpoint connection for writes - database.py: added _get_checkpoint_conn() helper - check-tls-cert.py: fixed SSLCertVerificationError.reason AttributeError 7. Unit tests - Test step bugs: - error_recovery_coverage_boost_steps.py: fixed invalid ULID _PLAN_ID - session_service_coverage_steps.py: fixed 'sha256:' prefix bug in checksum - database_models_new_coverage_steps.py: added 'name' field to session mock - async_audit_recording_steps.py: fixed Settings(audit_async=False) via env var - coverage_threshold_config_steps.py: added --coverage-min pattern support - m5_acms_smoke_steps.py: updated usage hint text - actor_cli_yaml_steps.py: updated 'Removed actor' → 'Actor removed' - aimodelscredentials_steps.py: set context.imported_class in import step - domain_base_model.feature: added missing 'When I examine model_config' step - tui_first_run_steps.py: fixed module reload to restore cleveragents.tui.* modules after test (prevented patch interference in subsequent tests) - tui_first_run_steps.py: added set_search('') step for empty string - resource_handler_base_coverage_r3_steps.py: use _MinimalHandler instead of DatabaseResourceHandler for NotImplementedError tests - resource_handler_crud.feature: updated to test new DatabaseHandler behavior - resource_handler_sandbox.feature: updated to test new DatabaseHandler behavior - tdd_json_decode_crash_persistence.feature: fixed @tdd_bug → @tdd_issue tags 8. Parallel test interference: - All step files using use_step_matcher('re') now reset to 'parse' at end to prevent global matcher state leaking to subsequent step files	2026-04-04 20:38:16 +00:00
freemo	70e2ce4386	Merge pull request 'fix(infra): resolve TLS handshake failure on git.dev.cleveragents.com' (#1865) from fix/infra-tls-handshake-failure-git-dev into master	2026-04-03 01:13:31 +00:00
freemo	8c81f13758	fix(infra): resolve TLS handshake failure on git.dev.cleveragents.com ... The TLS handshake failure on git.dev.cleveragents.com was caused by the hostname being absent from the certificate's Subject Alternative Names (SANs), or by SNI virtual-host misconfiguration on the server side. This commit delivers the repository-side remediation: - scripts/check-tls-cert.py: New TLS certificate health-check script. Connects to a hostname, verifies the certificate's SANs include the target hostname, checks expiry, and reports errors/warnings. Accepts an injectable SSLContext for unit testing without real network access. Supports wildcard SAN matching and configurable expiry warning threshold. - docs/development/ops-runbook.md: New ops runbook documenting the full certificate renewal procedure (Let's Encrypt/certbot and manual CA), SNI misconfiguration diagnosis steps, expiry monitoring with cron, and recommended alert thresholds (30/14/7/0 days). - features/tls_certificate_check.feature: 14 Behave scenarios tagged @tdd_issue @tdd_issue_1543 covering: missing SAN detection, valid SAN acceptance, expired certificate detection, expiry warning threshold, TLS handshake errors, connection timeouts, connection refused, wildcard SAN matching, and _hostname_matches_san unit tests. - features/steps/tls_certificate_check_steps.py: Step definitions for the above feature, using unittest.mock to inject SSL contexts and socket connections so no real network calls are made. - mkdocs.yml: Added Ops Runbook to the Development section navigation. The actual server-side certificate renewal (adding git.dev.cleveragents.com as a SAN and reloading the web server) must be performed by the server administrator following the procedure in docs/development/ops-runbook.md. Closes #1543 ISSUES CLOSED: #1543	2026-04-02 23:59:37 +00:00
freemo	37586882b3	chore(ci): extract behave-parallel runner script from noxfile.py into scripts/ ... Move the large embedded `_BEHAVE_PARALLEL_CLI_SOURCE` string constant out of noxfile.py and into a standalone `scripts/run_behave_parallel.py` module. The `_install_behave_parallel()` helper now reads the script from disk via `Path(__file__).parent / 'scripts' / 'run_behave_parallel.py'` instead of embedding the source as a raw string literal. This allows ruff to lint and type-check the runner independently, and makes noxfile.py significantly shorter and easier to read. No functional changes: the installed `behave-parallel` entry point is identical to the previous embedded version. Parallel and sequential modes, coverage integration, and the multiprocessing fork model are all preserved. Fixed two SIM105 lint violations in the extracted script (replaced try/except/pass with contextlib.suppress). ISSUES CLOSED: #1538	2026-04-02 23:44:30 +00:00
freemo	a8f7ed57cb	perf(tests): reduce per-feature startup cost with shared fixtures and lazy imports ... Created scripts/create_template_db.py that builds a pre-migrated SQLite template database using Base.metadata.create_all() + alembic stamp (~5ms for 34 tables, vs ~0.5-3s x 25 Alembic migrations per scenario). Nox unit_tests and coverage_report sessions generate the template before test execution and propagate CLEVERAGENTS_TEMPLATE_DB env var to all workers. features/environment.py before_all() installs a monkey-patch on MigrationRunner.init_or_upgrade that copies the template for fresh scenario temp DBs, falling through to real migrations for :memory:, existing files, and migration-runner unit tests. Quick wins: sleep(0.5) -> sleep(0.05) in cli_streaming wait step; removed redundant Background re-declaration in cli_streaming.feature scenario 7. ISSUES CLOSED: #483	2026-03-02 02:01:27 +00:00
brent.edwards	761db2518c	test: harden code sandbox, add path traversal protection, and fix test mocks	2026-02-12 01:07:44 +00:00
brent.edwards	7caac2714e	refactor: changes requested by Rui ... Removed `run-semgrep.sh` and automatically run setup-dev.sh	2026-02-11 00:59:58 +00:00
brent.edwards	7ef5ebb695	feat: Add Q0: Pre-commit hooks setup. ... This should automatically check for problems on build.	2026-02-10 16:04:17 +00:00
freemo	1486aa9de7	Removed sync and install scripts	2025-11-05 18:16:54 -05:00
freemo	9008f1a99d	Tentatively finishing phase 1	2025-11-05 17:07:35 -05:00
freemo	b941ef9a1a	Significantly revamped the base project to use more robust static checking including benchmarks and integration test support	2025-10-31 16:18:53 -04:00
freemo	f7a5d26c41	Modernized build tooling	2025-08-01 18:38:09 -04:00