cleveragents-core/docs/development/ci-cd.md

# CI/CD Pipeline and Branch Protection

This document describes the CI/CD pipeline, branch protection rules, and merge
requirements for the CleverAgents project. For quality tooling details (pre-commit
hooks, nox sessions, security scanning), see
[Quality Automation Guide](quality-automation.md).

## Branch Protection Rules

### Protected Branch: `master`

The `master` branch is the primary integration branch. All changes must go through
pull requests with the following protections enforced.

#### Required Status Checks

Every pull request targeting `master` must pass **all** of the following CI jobs
before merging is allowed:

| CI Job | What It Checks | Failure Means |
|--------|---------------|---------------|
| `lint` | Ruff format and lint | Code style violations or lint errors |
| `typecheck` | Pyright strict type checking | Type errors in `src/` |
| `security` | Bandit (HIGH severity gate) + Vulture dead code | Security vulnerabilities or dead code |
| `quality` | Radon complexity (grade F gate) | Extremely complex methods (31+ cyclomatic complexity) |
| `behave` | BDD unit tests (Python 3.11, 3.12, 3.13 matrix) | Failing unit test scenarios |
| `coverage` | Test coverage measurement | Coverage dropped below 85% |
| `build` | Wheel build | Build failure |

**Deployment-gating jobs** (run after core checks pass):

| CI Job | Depends On | What It Checks |
|--------|-----------|----------------|
| `docker` | `lint`, `typecheck`, `behave`, `security` | Docker image builds and runs |
| `helm` | `lint`, `typecheck`, `behave`, `security` | Helm chart lints and templates |

#### Required Reviews

- **Minimum 1 approving review** required before merge.
- Reviews are selective (see [Review Priority Matrix](#review-priority-matrix) below).
- Stale reviews are dismissed when new commits are pushed.
- Code owners can be configured per directory if needed.

#### Additional Protections

- **No direct pushes** to `master`. All changes must come via pull request.
  The `no-commit-to-branch` pre-commit hook enforces this locally.
- **No force pushes** to `master`. History must be preserved.
- **No deletions** of the `master` branch.
- **Require branches to be up-to-date** before merging (prevents merge skew).

### Setting Up Branch Protection in Forgejo

To configure these rules in Forgejo:

1. Navigate to **Repository Settings** > **Branches** > **Branch Protection**.
2. Add a rule for branch name pattern: `master`.
3. Enable the following settings:

```
[x] Enable branch protection
[x] Block push (no direct pushes)
[x] Block force push
[x] Block branch deletion
[x] Require pull request reviews before merging
    - Required approvals: 1
    - Dismiss stale reviews: Yes
[x] Require status checks to pass before merging
    - Required checks:
      - lint
      - typecheck
      - security
      - quality
      - behave
      - coverage
      - build
[x] Require branches to be up-to-date before merging
```

4. Save the branch protection rule.

## Review Process

### Review Priority Matrix

Not all code changes require the same level of review attention. Use this matrix
to determine review depth:

| Priority | Category | Review Depth | Examples |
|----------|----------|-------------|----------|
| **P0** | Architecture and Security | Deep review required | Service layer design, DB schema, API contracts, sandbox boundaries, authentication |
| **P1** | Complex Business Logic | Careful review | Decision tree traversal, merge conflict resolution, dependency closure, state machines |
| **P2** | Normal Features | Standard review | CLI commands, model definitions, straightforward service methods |
| **P3** | Tests, Docs, Refactoring | Trust automation | Behave scenarios, documentation updates, import reorganization, formatting |

### What Reviewers Should Focus On

**Always review** (P0 items):

- Service layer design choices and dependency injection patterns
- Database schema decisions and Alembic migrations
- API contract definitions and interface boundaries
- Sandbox security boundaries and isolation guarantees
- Input validation and sanitization logic
- Any code touching `eval()`, `exec()`, `compile()`, or `subprocess`

**Review carefully** (P1 items):

- Algorithm implementations (decision tree, merge, closure computation)
- State machine transitions and lifecycle management
- Error propagation across actor and plan boundaries
- Concurrent access patterns and thread safety

**Standard review** (P2 items):

- Verify tests exist for new functionality
- Check that argument validation follows `CONTRIBUTING.md` guidelines
- Confirm type annotations are complete (no bare `Any`)

**Trust automation** (P3 items):

- If `nox` passes (lint, typecheck, tests, coverage, security), formatting and
  style are covered.
- Behave test scenarios and Robot integration tests follow established patterns.
- Documentation changes are verified by `nox -s docs` (MkDocs build).

### Review Checklist

Every PR should pass this checklist (also available in the
[PR template](../../.forgejo/pull_request_template.md)):

- [ ] Code follows `CONTRIBUTING.md` coding standards
- [ ] All public/protected methods have argument validation
- [ ] Static typing is complete (no bare `Any` unless justified)
- [ ] `nox -s typecheck` passes
- [ ] `nox -s lint` passes
- [ ] Unit tests written/updated (Behave scenarios in `features/`)
- [ ] Integration tests written/updated (Robot suites in `robot/`) if applicable
- [ ] Coverage remains above 85%
- [ ] No security issues introduced
- [ ] No dead code introduced
- [ ] Documentation updated if behavior changed

## CI Pipeline Reference

### Workflow Files

| Workflow | File | Trigger | Purpose |
|----------|------|---------|---------|
| CI | `.forgejo/workflows/ci.yml` | Push to `main`/`develop`, PRs to `main` | Full validation pipeline |
| Nightly Quality | `.forgejo/workflows/nightly-quality.yml` | Midnight UTC (cron), manual | Comprehensive quality monitoring |

### CI Job Dependency Graph

```
lint ──────────┐
typecheck ─────┤
               ├── coverage (needs lint + typecheck)
security ──────┤
               ├── docker (needs lint + typecheck + behave + security)
behave ────────┤
               └── helm   (needs lint + typecheck + behave + security)
quality ───────── (independent)
build ─────────── (independent)
```

### Quality Gates Summary

All gates must pass for a PR to be mergeable:

| Gate | Threshold | Enforced By |
|------|-----------|-------------|
| Formatting | Zero violations | `lint` job (ruff format) |
| Linting | Zero violations | `lint` job (ruff check) |
| Type Safety | Zero errors | `typecheck` job (pyright strict) |
| Security | Zero HIGH severity | `security` job (bandit) |
| Dead Code | Zero findings | `security` job (vulture) |
| Complexity | No grade-F methods | `quality` job (radon) |
| Unit Tests | All pass (3.11-3.13) | `behave` job |
| Coverage | >= 85% | `coverage` job |
| Build | Wheel builds | `build` job |

### Nightly Quality Monitoring

The nightly workflow provides trend data beyond PR-level checks:

- Full lint, type check, and security scan (all severities, not just HIGH)
- Complete Behave test suite with coverage measurement
- Complexity and maintainability index analysis
- Quality trend JSON with timestamps (90-day artifact retention)

Reports are uploaded as artifacts under `nightly-quality-reports` and can be
downloaded from the Forgejo Actions UI.

## Local Development Workflow

### Before Pushing a PR

```bash
# Quick validation (runs default nox sessions)
nox

# Or run individual checks
nox -s lint              # Formatting and linting
nox -s typecheck         # Type checking
nox -s unit_tests        # Behave tests
nox -s coverage_report   # Coverage (must be >85%)
nox -s security_scan     # Security scanning
```

### Pre-commit Hooks

Pre-commit hooks run automatically on `git commit` and catch most issues before
they reach CI. See [Quality Automation Guide](quality-automation.md) for the
full list of hooks.

If hooks are not installed:

```bash
bash scripts/setup-dev.sh
# Or manually:
pre-commit install
pre-commit install --hook-type commit-msg
```