From 1031fd0fb18468aad3494d0f856b84cd414eff3f Mon Sep 17 00:00:00 2001 From: CleverThis Date: Tue, 14 Apr 2026 05:25:30 +0000 Subject: [PATCH 1/2] fix(agents): make bug-hunt-pool-supervisor tracking non-blocking to prevent initialization hangs --- .opencode/agents/bug-hunt-pool-supervisor.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.opencode/agents/bug-hunt-pool-supervisor.md b/.opencode/agents/bug-hunt-pool-supervisor.md index 101013c22..161bddef4 100644 --- a/.opencode/agents/bug-hunt-pool-supervisor.md +++ b/.opencode/agents/bug-hunt-pool-supervisor.md @@ -91,7 +91,7 @@ Each cycle: 4. **Monitor workers.** Count active workers, check for stuck sessions. -5. **Update tracking.** Every 3 cycles, create a status tracking issue via `automation-tracking-manager` with prefix `AUTO-BUG-POOL`. +5. **Update tracking (non-blocking).** Every 3 cycles, attempt to create a status tracking issue via `automation-tracking-manager` with prefix `AUTO-BUG-POOL`. This step is **best-effort** — if the call does not complete within a reasonable time or fails, skip it and continue to the next cycle. **Never block the main loop waiting for tracking.** Tracking is informational only; the supervisor's core function (module scanning and worker dispatch) must continue regardless. ## Finding Validation Gate @@ -126,3 +126,4 @@ Supervisor: Bug Hunt Pool | Agent: bug-hunt-pool-supervisor 7. **Apply labels via `forgejo-label-manager`.** Never apply labels directly or using the Forgejo MCP/task. All label operations must go through `forgejo-label-manager`. 8. **Exhaustive pagination for all list results.** Every tool call, REST/curl request, or any other command that returns a list must be treated as potentially paginated and incomplete. Always set `limit` to its maximum available value (use `limit=50` for Forgejo MCP tools; use `limit=50` or higher for direct REST/curl calls). After each list response, check whether the number of returned items equals the page size — if so, there are likely more results; fetch the next page (`page=2`, `page=3`, …) and continue until receiving a partial page. Never assume the first response is the complete result. This rule applies to every list-returning call without exception. *Examples specific to this agent (not exhaustive):* `forgejo_list_repo_issues` (default 20 — use `limit=50` and paginate to check for all existing bug issues before filing duplicates); `forgejo_list_repo_pull_requests` (same — check all open PRs that may already address a discovered bug). +9. **Tracking is non-blocking.** The `automation-tracking-manager` call in step 5 must never block the main loop. If it hangs or fails, skip it and proceed. Core functionality (module mapping, worker dispatch, monitoring) takes priority over status reporting. -- 2.52.0 From 1e543e8fa3182c540764a7d0dc2840e8df3cea3c Mon Sep 17 00:00:00 2001 From: CleverThis Date: Tue, 14 Apr 2026 08:12:26 +0000 Subject: [PATCH 2/2] docs(development): add developer setup guide Add comprehensive developer setup guide covering prerequisites, development workflow, testing, linting, type checking, commit guidelines, devcontainer setup, and troubleshooting. Refs: #9123 --- CHANGELOG.md | 1 + docs/development/setup.md | 521 ++++++++++++++++++++++++++++++++++++++ mkdocs.yml | 1 + 3 files changed, 523 insertions(+) create mode 100644 docs/development/setup.md diff --git a/CHANGELOG.md b/CHANGELOG.md index 0cf791e12..3f892c969 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ### Added +- **Developer Setup Guide** (#9123): Added comprehensive developer setup guide at `docs/development/setup.md` covering prerequisites, development workflow, testing, linting, and commit guidelines. - **TDD Issue-Capture Test Activation** (#7025): Replaced 234 bare `@skip` tags across 82 Behave feature files with the correct `@tdd_expected_fail @tdd_issue @tdd_issue_` tag system. Scenarios whose referenced bugs were already fixed diff --git a/docs/development/setup.md b/docs/development/setup.md new file mode 100644 index 000000000..bc2da45ad --- /dev/null +++ b/docs/development/setup.md @@ -0,0 +1,521 @@ +# Developer Setup Guide + +This guide walks you through setting up a local development environment for +**cleveragents-core** from scratch. By the end you will have a fully working +environment with all dependencies installed, tests passing, and pre-commit hooks +active. + +--- + +## 1. Prerequisites + +Install the following tools before cloning the repository. + +### Python 3.13+ + +CleverAgents requires **Python 3.13** or later. + +```bash +# Verify your installed version +python3 --version # must print Python 3.13.x or higher +``` + +Recommended installation methods: + +| Platform | Method | +|----------|--------| +| macOS | `brew install python@3.13` or [python.org](https://www.python.org/downloads/) | +| Linux (Debian/Ubuntu) | `sudo apt install python3.13 python3.13-venv python3.13-dev` | +| Linux (Fedora/RHEL) | `sudo dnf install python3.13` | +| Windows | [python.org installer](https://www.python.org/downloads/) (add to PATH) | +| Any | [pyenv](https://github.com/pyenv/pyenv): `pyenv install 3.13` | + +### uv + +[uv](https://docs.astral.sh/uv/) is the project's fast Python package manager and +virtual-environment tool. + +```bash +# macOS / Linux +curl -LsSf https://astral.sh/uv/install.sh | sh + +# Windows (PowerShell) +powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex" + +# Verify +uv --version +``` + +### nox + +[nox](https://nox.thea.codes/) is the project's task runner. All test, lint, +type-check, and build commands are executed through `nox` sessions — **never** +invoke `behave`, `ruff`, `pyright`, or other tools directly. + +```bash +pip install nox +# or, using uv: +uv tool install nox +``` + +### Git + +Git 2.30+ is required. Most systems ship a recent enough version. + +```bash +git --version # must be 2.30 or later +``` + +### Commitizen (optional but recommended) + +[Commitizen](https://commitizen.github.io/cz-cli/) guides you through writing +[Conventional Changelog](https://github.com/conventional-changelog/conventional-changelog-eslint/blob/master/convention.md)-compliant +commit messages interactively. All commits **must** follow this standard. + +```bash +npm install -g commitizen@2.8.6 cz-customizable@4.0.0 +``` + +> **Note:** npm is only needed for Commitizen. Node.js is not otherwise required +> to develop or run cleveragents-core. + +### pre-commit + +[pre-commit](https://pre-commit.com/) runs automated checks before each commit. +It is installed as part of the dev dependencies (see [Getting Started](#2-getting-started)) +but the CLI must also be available on your PATH. + +```bash +pip install pre-commit +# or, using uv: +uv tool install pre-commit +``` + +--- + +## 2. Getting Started + +### Clone the Repository + +```bash +git clone https://git.cleverthis.com/cleveragents/cleveragents-core.git +cd cleveragents-core +``` + +### Create and Activate a Virtual Environment + +```bash +# Create a venv with Python 3.13 +uv venv --python 3.13 + +# Activate (macOS / Linux) +source .venv/bin/activate + +# Activate (Windows PowerShell) +.venv\Scripts\Activate.ps1 +``` + +### Install All Dependencies + +Install the package in editable mode together with all development and test +extras: + +```bash +uv pip install -e ".[dev,tests,docs]" +``` + +This installs: + +- **Runtime** dependencies (typer, pydantic, langchain, etc.) +- **Dev** extras: ruff, pyright, behave, pytest, pre-commit, bandit, semgrep, + vulture, radon +- **Tests** extras: behave, slipcover, asv, robotframework, robotframework-pabot +- **Docs** extras: mkdocs, mkdocs-material, mkdocstrings + +### Install pre-commit Hooks + +```bash +pre-commit install +``` + +This installs Git hooks that run linting and formatting checks automatically on +`git commit`. + +### Verify the Installation + +Run the full default nox pipeline to confirm everything is working: + +```bash +nox +``` + +This executes (in order): `lint` → `typecheck` → `unit_tests` → +`integration_tests` → `coverage_report`. A clean run with all sessions passing +means your environment is correctly set up. + +--- + +## 3. Development Workflow + +All quality-gate commands are run through `nox`. The `noxfile.py` in the +repository root defines every available session. + +### Running Tests + +```bash +# Run all default sessions +nox + +# Unit tests only (Behave / BDD) +nox -s unit_tests + +# Integration tests only (Robot Framework) +nox -s integration_tests + +# Coverage report — enforces ≥ 97% threshold +nox -s coverage_report + +# Run a single Behave feature file +nox -s unit_tests -- features/plan_model.feature + +# Run scenarios with a specific tag +nox -s unit_tests -- --tags=@smoke + +# Run a single Robot suite +nox -s integration_tests -- --suite robot/plan_persistence_e2e.robot + +# Performance benchmarks (ASV) +nox -s benchmark +``` + +> See [Testing Guide](testing.md) for full details on the testing strategy, +> coverage requirements, and test suite structure. + +### Linting + +```bash +# Check for lint violations (ruff) +nox -s lint + +# Auto-fix safe violations +nox -s lint -- --fix + +# Check formatting only (no changes) +nox -s format -- --check + +# Apply formatting +nox -s format +``` + +Ruff is configured in `pyproject.toml` under `[tool.ruff]`. The project targets +Python 3.13 with an 88-character line length and enforces rules from the `E`, +`F`, `W`, `B`, `UP`, `I`, `SIM`, and `RUF` rule sets. + +### Type Checking + +```bash +# Run pyright type checker +nox -s typecheck +``` + +Pyright is configured in `pyproject.toml` under `[tool.pyright]`. All source +files under `src/` must pass strict type checking. Do not add `# type: ignore` +comments without a documented justification. + +### Security Scanning + +```bash +# Run bandit + vulture +nox -s security_scan + +# Complexity metrics (radon) +nox -s complexity +``` + +### Building Documentation + +```bash +# Build the MkDocs site locally +nox -s docs + +# Serve docs with live reload at http://127.0.0.1:8000 +nox -s docs -- serve +``` + +### Full Quality Gate (pre-PR checklist) + +Before opening a pull request, run the complete quality gate: + +```bash +nox +``` + +All sessions must pass. The CI pipeline enforces the same checks and will block +merges on any failure. + +--- + +## 4. Commit Guidelines + +### Conventional Changelog Format + +Every commit **must** follow the +[Conventional Changelog standard](https://github.com/conventional-changelog/conventional-changelog-eslint/blob/master/convention.md). +The format is: + +``` +(): + + + + +``` + +**Allowed types:** + +| Type | When to use | +|------|-------------| +| `feat` | New feature or user-visible behaviour | +| `fix` | Bug fix | +| `docs` | Documentation only | +| `refactor` | Code restructuring without behaviour change | +| `test` | Adding or updating tests | +| `chore` | Build system, tooling, dependency updates | +| `perf` | Performance improvement | +| `ci` | CI/CD pipeline changes | + +**Example:** + +``` +fix(concurrency): guard execute_plan with advisory lock + +LockService was implemented but never wired into the plan execution +path, leaving execute_plan() unprotected against concurrent calls. + +ISSUES CLOSED: #7989 +``` + +### Using Commitizen (recommended) + +```bash +# Instead of `git commit`, use: +git cz +``` + +Commitizen walks you through an interactive prompt to build a correctly +formatted commit message. All CleverThis repositories ship a local +`cz-customizable` configuration. + +### Atomic Commits + +- **One logical change per commit.** Never bundle unrelated changes. +- **Include tests with the change.** Feature + tests are one logical unit. +- **Include documentation with the change.** Update relevant docs in the same commit. +- **Each commit must build and pass all tests.** The repository must be in a + working state at every commit in history. + +See [CONTRIBUTING.md](../../CONTRIBUTING.md) for the full commit quality policy. + +### Branch Naming + +Use the following prefixes: + +| Prefix | Purpose | +|--------|---------| +| `feat/-` | New feature | +| `fix/-` | Bug fix | +| `docs/` | Documentation | +| `refactor/` | Refactoring | +| `chore/` | Tooling / maintenance | +| `test/` | Test-only changes | + +Examples: + +``` +feat/issue-1234-plan-correction-rollback +fix/7989-lock-service-concurrency +docs/developer-setup-guide +``` + +--- + +## 5. Devcontainer Setup + +The repository ships a [Dev Container](https://containers.dev/) configuration +that provides a fully pre-configured development environment with Python 3.13, +uv, nox, and all dependencies pre-installed. This is the **recommended** setup +for contributors who want zero-friction onboarding. + +### Requirements + +- [Docker Desktop](https://www.docker.com/products/docker-desktop/) (macOS / + Windows) or Docker Engine (Linux) +- [VS Code](https://code.visualstudio.com/) with the + [Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers), + **or** the [devcontainer CLI](https://github.com/devcontainers/cli) + +### Opening in VS Code + +1. Open the repository folder in VS Code. +2. When prompted *"Reopen in Container"*, click **Reopen in Container**. + Alternatively: open the Command Palette (`Ctrl+Shift+P` / `Cmd+Shift+P`) and + run **Dev Containers: Reopen in Container**. +3. VS Code builds the container image and installs all extensions. This takes a + few minutes on first run; subsequent opens are fast. +4. A terminal inside the container is ready to use. All tools (`uv`, `nox`, + `git`, `pre-commit`) are on the PATH. + +### Using the devcontainer CLI + +```bash +# Start the container +devcontainer up --workspace-folder . + +# Execute a command inside the container +devcontainer exec --workspace-folder . nox -s unit_tests +``` + +### Named Configurations + +The repository may define multiple named devcontainer configurations (e.g., +`default`, `docs-only`). To select one in VS Code, use +**Dev Containers: Reopen in Container…** and choose the desired configuration +from the list. + +### Environment Variables + +Sensitive configuration (API keys, database URLs) should be placed in a +`.env` file at the repository root. The devcontainer automatically mounts +this file. **Never commit `.env` to version control.** + +Common variables: + +| Variable | Description | +|----------|-------------| +| `CLEVERAGENTS_DATABASE_URL` | SQLite or PostgreSQL connection string | +| `CLEVERAGENTS_HOME` | Override the default data directory | +| `CLEVERAGENTS_TESTING_USE_MOCK_AI` | Set to `1` to use mock AI in tests | +| `OPENAI_API_KEY` | OpenAI provider key (optional) | +| `ANTHROPIC_API_KEY` | Anthropic provider key (optional) | + +--- + +## 6. Troubleshooting + +### `python3 --version` shows < 3.13 + +Your system Python is too old. Install Python 3.13 via pyenv, your OS package +manager, or [python.org](https://www.python.org/downloads/), then recreate the +virtual environment: + +```bash +uv venv --python 3.13 +source .venv/bin/activate +uv pip install -e ".[dev,tests,docs]" +``` + +### `nox` command not found + +Install nox into your system Python or via uv tools: + +```bash +pip install nox +# or +uv tool install nox +``` + +Ensure the tool installation directory is on your `PATH` (uv prints the path +after installation). + +### `pre-commit` hooks not running + +Reinstall the hooks: + +```bash +pre-commit install --overwrite +``` + +If hooks still do not run, verify that `.git/hooks/pre-commit` exists and is +executable. + +### Import errors after installing dependencies + +Ensure you activated the virtual environment before running any commands: + +```bash +source .venv/bin/activate # macOS / Linux +.venv\Scripts\Activate.ps1 # Windows +``` + +Then verify the package is installed in editable mode: + +```bash +pip show cleveragents +``` + +### `nox -s unit_tests` fails with `AmbiguousStep` + +Two Behave step files define the same step pattern. Check the error message for +the conflicting step text, then either rename one step or add a unique prefix. +See the [Testing Guide](testing.md#unit-tests-behave) for step naming conventions. + +### Coverage below 97% + +Run the HTML coverage report to identify uncovered lines: + +```bash +nox -s coverage_report +open build/htmlcov/index.html +``` + +Write Behave scenarios targeting the uncovered code paths. Do **not** lower the +threshold or add `# pragma: no cover` without explicit team agreement. + +### `pyright` reports errors in generated or third-party stubs + +Check `pyproject.toml` under `[tool.pyright]` for the `exclude` list. Generated +stubs under `docs/reference/contracts/stubs/` are excluded by the ruff +configuration and should also be excluded from pyright. Add a targeted +`# type: ignore` with a comment only as a last resort. + +### Robot Framework `Resource file does not exist` + +Always use `${CURDIR}/` prefix for `Resource` imports: + +```robot +Resource ${CURDIR}/common.resource +``` + +Bare paths are resolved relative to the working directory, not the test file. + +### Devcontainer fails to start + +1. Ensure Docker is running (`docker info`). +2. Try **Dev Containers: Rebuild Container** from the VS Code Command Palette. +3. Check the Docker build log in the VS Code Output panel for errors. +4. If the issue persists, delete the container image and rebuild: + ```bash + docker system prune -f + devcontainer up --workspace-folder . --remove-existing-container + ``` + +### SQLite in-memory tests lose data between steps + +When writing Behave tests with `sqlite:///:memory:`, use a single shared +`Session` across all repository calls in a scenario — do not create a new +session per call. See the [Testing Guide](testing.md#troubleshooting) for the +correct pattern. + +--- + +## See Also + +- [Testing Guide](testing.md) — Full testing strategy, coverage requirements, + and suite documentation +- [CI/CD Pipeline](ci-cd.md) — Forgejo CI job definitions and pipeline stages +- [Quality Automation](quality-automation.md) — Automated quality gates and + enforcement +- [CONTRIBUTING.md](../../CONTRIBUTING.md) — Commit standards, code of conduct, + and contribution workflow +- [Architecture Decision Records](../adr/index.md) — Key design decisions + (ADR-003 DI, ADR-005 Tech Stack, ADR-009 Project Model, etc.) diff --git a/mkdocs.yml b/mkdocs.yml index 76940184b..f8dbf1bb7 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -30,6 +30,7 @@ nav: - ACMS Context Hydration: modules/context-hydration.md - Git Worktree Sandbox: modules/git-worktree-sandbox.md - Development: + - Setup Guide: development/setup.md - Agent System Specification: development/agent-system-specification.md - CI/CD Pipeline: development/ci-cd.md - Quality Automation: development/quality-automation.md -- 2.52.0