# CI/CD Pipeline — CleverAgents Project

> ⚠️ Rules here override `cleverthis-guidelines`. Apply these exactly.

---

## Workflow Files

All pipeline definitions live in `.forgejo/workflows/`.

| Workflow | Trigger | Purpose |
|----------|---------|---------|
| `ci.yml` | Push to `master`/`develop`; PRs to `master`/`develop*` | Full CI: lint, typecheck, security, quality, tests, coverage, benchmarks, build, Docker |
| `nightly-quality.yml` | Daily at midnight UTC; manual dispatch | Extended quality sweep with complexity and maintainability analysis |
| `release.yml` | Push of `v*` tags (e.g. `v3.6.0`) | Build wheel, build and push Docker image, create Forgejo release with artifacts |

---

## All CI Jobs (ci.yml)

Jobs without explicit dependencies run in parallel.

| Job | Depends On | nox Session(s) | Description |
|-----|-----------|----------------|-------------|
| `lint` | — | `nox -s lint` + `nox -s format -- --check` | Ruff linting + format check |
| `typecheck` | — | `nox -s typecheck` | Pyright strict type checking |
| `security` | — | `nox -s security_scan` + `nox -s dead_code` | bandit + semgrep + vulture |
| `quality` | — | `nox -s complexity` | Radon complexity analysis |
| `unit_tests` | — | `nox -s unit_tests` | Behave BDD tests |
| `integration_tests` | — | `nox -s integration_tests` | Robot Framework via pabot |
| `e2e_tests` | — | `nox -s e2e_tests` | Robot Framework with real LLM keys |
| `coverage` | `lint`, `typecheck` | `nox -s coverage_report` | Slipcover (fail-under 97%) |
| `benchmark-regression` | `lint`, `typecheck` | `nox -s benchmark_regression` | ASV benchmark regression on PRs |
| `benchmark-publish` | — | `nox -s benchmark` (publish) | ASV publish (push to master/develop) |
| `build` | — | `nox -s build` | Wheel distribution build |
| `docker` | `lint`, `typecheck`, `unit_tests`, `security` | (Docker CLI) | Docker image build + smoke test |
| `status-check` | All required jobs | — | Consolidation gate for branch protection |

---

## Required Checks for Merge

The `status-check` job is the **single branch-protection gate**. It depends on
all required jobs and fails if any of them did not succeed.

**These five jobs MUST pass — PR cannot be merged otherwise:**

| Job | What it checks |
|-----|----------------|
| `lint` | Ruff linting passes; code is formatted per ruff standards |
| `typecheck` | Pyright strict passes; no type errors; no `# type: ignore` |
| `security` | No high-severity bandit findings; semgrep rules pass; no dead code |
| `unit_tests` | All Behave BDD scenarios pass; TDD tag gates pass |
| `coverage` | Slipcover reports ≥ 97% test coverage |

**Informational only (not blocking):**
- `integration_tests`
- `e2e_tests`
- `benchmark-regression`

---

## How to Read CI Results

1. Open the PR → scroll to **Checks** section
2. Each job is a separate named check — click any failed job for full logs
3. `status-check` provides a consolidated summary of all required results
4. For coverage: look for `COVERAGE OK` or `COVERAGE FAILED` in coverage job logs
5. Benchmark regression: results uploaded as an artifact for manual review

---

## Release Process

```bash
# Trigger a release by pushing a version tag:
git tag v<MAJOR>.<MINOR>.<PATCH>      # e.g. git tag v1.2.3
git push origin v<MAJOR>.<MINOR>.<PATCH>
```

The `release.yml` workflow then runs automatically:

1. **Builds the wheel** via `nox -s build` → uploaded as artifact
2. **Builds the Docker image** tagged with the version and `latest`
3. **Pushes the Docker image** to the configured container registry
4. **Creates a Forgejo release** with the wheel attached as a downloadable artifact

### If the release workflow fails

1. Verify all required secrets are configured (see table below)
2. Confirm `nox -s build` passes locally
3. Delete the tag: `git tag -d vX.Y.Z && git push origin :refs/tags/vX.Y.Z`
4. Fix the issue in master, then re-tag and re-push

### Required repository secrets

| Secret | Purpose | Example value |
|--------|---------|---------------|
| `CONTAINER_REGISTRY` | Docker registry URL | `registry.cleverthis.com/cleveragents/core` |
| `CONTAINER_REGISTRY_USER` | Registry auth username | — |
| `CONTAINER_REGISTRY_PASSWORD` | Registry auth password | — |
| `FORGEJO_TOKEN` | API token for creating releases | — |
| `FORGEJO_URL` | Forgejo instance URL | `https://git.cleverthis.com` |

---

## Dependency Caching

- CI uses `actions/cache@v3` to cache uv package downloads between runs
- Cache key: `pyproject.toml` hash
- Caches are automatically invalidated when `pyproject.toml` changes
- Reduces CI run time by avoiding redundant package downloads

---

## Nightly Quality Sweep

`nightly-quality.yml` runs daily at midnight UTC (and can be dispatched manually):

- Extended quality analysis: full complexity and maintainability metrics via Radon
- Corresponds to `nox -s complexity`
- Results surface issues not caught by standard CI (complexity drift, maintainability index)

---

## Branch Protection

- The `master` branch is protected
- The `status-check` job is the required check
- Direct pushes to `master` are prohibited — all changes via PR
- At least 1 approving review required before merge (self-approval permitted)
