cleveragents-core/.opencode/skills/cleverthis-guidelines/references/testing

Testing Guidelines

Language / tooling note: This project uses Python with nox, Behave, and Robot Framework. All nox -s <session> commands below are the project-specific examples. Replace with the equivalent task runner command for your language ecosystem.

Testing Philosophy

• BDD (Behavior-Driven Development) for ALL unit-level tests
• Project framework: Behave (Cucumber/Gherkin standard); other options: Cucumber, SpecFlow, pytest-bdd, rspec, godog
• DO NOT write xUnit-style tests (raw assertions without BDD structure)
• All unit tests = Gherkin feature files in the designated unit test directory (project: features/)
• Integration tests: separate framework exercising real services (project: Robot Framework via pabot)
• Test-first development: write tests BEFORE implementation
• Testing is non-optional — part of definition of done

---

BDD Unit Tests (project uses Behave)

Location: designated unit test directory (project: features/)

• Feature files in Gherkin syntax (.feature)
• Step definitions grouped by feature (name step file after feature)
• Add to existing step files before creating new ones
• Never commit placeholder steps (all steps must be implemented)
• Never use empty/stub implementations in step definitions (no no-op bodies)
• Run via task runner: nox -s unit_tests

---

Integration Tests (project uses Robot Framework)

• Framework: Robot Framework via pabot (parallel execution)
• Location: integration test directory (project: robot/)
• Exercise real services, real endpoints, real dependencies
• NO mocking in integration tests (strictly prohibited)
• Mocks acceptable only for truly impractical external dependencies
• Must be updated when component interfaces change
• Run via task runner: nox -s integration_tests

---

Coverage Requirement

• Project threshold: ≥ 97% (overrides C.O.C. baseline of 85%)
• Measured independently for unit AND integration suites
• Run via task runner: nox -s coverage_report
• PR blocked automatically if below threshold
• Project owner can grant exception (must be documented)
• Excluding code from coverage: explicit project owner approval required

---

Mock Placement Rules (strict)

• Mocks/fakes/stubs/fixtures: designated mock directory ONLY (project: features/mocks/)
• NEVER in production source directory (src/ or equivalent)
• NEVER in scripts/
• Production code must NOT contain if testing: guards or test-only paths
• Use dependency injection to substitute test doubles
• Mocks allowed ONLY in unit tests
• Integration tests must use real services/endpoints/dependencies

---

What Must Have Tests

• Every non-trivial function, method, class (regardless of overall project coverage)
• Both normal operation AND error/exception paths
• Edge cases and failure modes
• All newly introduced behavior
• Regression tests for every bug fix (via mandatory TDD workflow)

---

TDD Bug Fix Workflow

1. Create Type/Testing "TDD:" issue before writing any code
2. Write BDD scenario that captures the buggy behavior (project: Behave)
3. Tag with: @tdd_issue, @tdd_issue_N, @tdd_expected_fail
4. @tdd_expected_fail inverts pass/fail — CI passes even though bug exists
5. Merge TDD test PR to master (proves bug, CI passes)
6. Implement fix on bugfix/ branch
7. REMOVE @tdd_expected_fail tag (leave @tdd_issue and @tdd_issue_N)
8. Test must now pass normally
9. CI quality gate checks @tdd_expected_fail is ABSENT
10. Merge fix PR → close bug issue
11. Test remains permanently as regression guard

Assertion type requirement: Expected-fail steps must fail via the language's assertion type — NOT runtime/infrastructure exceptions.

• Python: assert condition or raise AssertionError(...) — not ValueError/RuntimeError
• Java: throw new AssertionError(...) — not RuntimeException
• JavaScript/TypeScript: throw new assert.AssertionError(...) — not Error/TypeError

---

Running Tests

# Always use the task runner — never invoke frameworks directly
# Python/nox examples (replace with your ecosystem's equivalent):

nox                           # run all default sessions
nox -s unit_tests             # BDD unit tests only (Behave)
nox -s integration_tests      # integration tests (Robot Framework)
nox -s coverage_report        # generate coverage report
nox -s lint                   # linting + format check (ruff)
nox -s typecheck              # static type checking (Pyright)
nox -s security_scan          # security scan (bandit + semgrep + vulture)
nox -s dead_code              # dead code detection (vulture)
nox -s format                 # auto-format (ruff)

If a task runner session is missing → add it to the task runner configuration FIRST.