Compare commits
2 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 1e543e8fa3 | |||
| 1031fd0fb1 |
@@ -91,7 +91,7 @@ Each cycle:
|
||||
|
||||
4. **Monitor workers.** Count active workers, check for stuck sessions.
|
||||
|
||||
5. **Update tracking.** Every 3 cycles, create a status tracking issue via `automation-tracking-manager` with prefix `AUTO-BUG-POOL`.
|
||||
5. **Update tracking (non-blocking).** Every 3 cycles, attempt to create a status tracking issue via `automation-tracking-manager` with prefix `AUTO-BUG-POOL`. This step is **best-effort** — if the call does not complete within a reasonable time or fails, skip it and continue to the next cycle. **Never block the main loop waiting for tracking.** Tracking is informational only; the supervisor's core function (module scanning and worker dispatch) must continue regardless.
|
||||
|
||||
## Finding Validation Gate
|
||||
|
||||
@@ -126,3 +126,4 @@ Supervisor: Bug Hunt Pool | Agent: bug-hunt-pool-supervisor
|
||||
|
||||
7. **Apply labels via `forgejo-label-manager`.** Never apply labels directly or using the Forgejo MCP/task. All label operations must go through `forgejo-label-manager`.
|
||||
8. **Exhaustive pagination for all list results.** Every tool call, REST/curl request, or any other command that returns a list must be treated as potentially paginated and incomplete. Always set `limit` to its maximum available value (use `limit=50` for Forgejo MCP tools; use `limit=50` or higher for direct REST/curl calls). After each list response, check whether the number of returned items equals the page size — if so, there are likely more results; fetch the next page (`page=2`, `page=3`, …) and continue until receiving a partial page. Never assume the first response is the complete result. This rule applies to every list-returning call without exception. *Examples specific to this agent (not exhaustive):* `forgejo_list_repo_issues` (default 20 — use `limit=50` and paginate to check for all existing bug issues before filing duplicates); `forgejo_list_repo_pull_requests` (same — check all open PRs that may already address a discovered bug).
|
||||
9. **Tracking is non-blocking.** The `automation-tracking-manager` call in step 5 must never block the main loop. If it hangs or fails, skip it and proceed. Core functionality (module mapping, worker dispatch, monitoring) takes priority over status reporting.
|
||||
|
||||
@@ -17,6 +17,7 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
### Added
|
||||
|
||||
- **Developer Setup Guide** (#9123): Added comprehensive developer setup guide at `docs/development/setup.md` covering prerequisites, development workflow, testing, linting, and commit guidelines.
|
||||
- **TDD Issue-Capture Test Activation** (#7025): Replaced 234 bare `@skip` tags
|
||||
across 82 Behave feature files with the correct `@tdd_expected_fail @tdd_issue
|
||||
@tdd_issue_<N>` tag system. Scenarios whose referenced bugs were already fixed
|
||||
|
||||
@@ -0,0 +1,521 @@
|
||||
# Developer Setup Guide
|
||||
|
||||
This guide walks you through setting up a local development environment for
|
||||
**cleveragents-core** from scratch. By the end you will have a fully working
|
||||
environment with all dependencies installed, tests passing, and pre-commit hooks
|
||||
active.
|
||||
|
||||
---
|
||||
|
||||
## 1. Prerequisites
|
||||
|
||||
Install the following tools before cloning the repository.
|
||||
|
||||
### Python 3.13+
|
||||
|
||||
CleverAgents requires **Python 3.13** or later.
|
||||
|
||||
```bash
|
||||
# Verify your installed version
|
||||
python3 --version # must print Python 3.13.x or higher
|
||||
```
|
||||
|
||||
Recommended installation methods:
|
||||
|
||||
| Platform | Method |
|
||||
|----------|--------|
|
||||
| macOS | `brew install python@3.13` or [python.org](https://www.python.org/downloads/) |
|
||||
| Linux (Debian/Ubuntu) | `sudo apt install python3.13 python3.13-venv python3.13-dev` |
|
||||
| Linux (Fedora/RHEL) | `sudo dnf install python3.13` |
|
||||
| Windows | [python.org installer](https://www.python.org/downloads/) (add to PATH) |
|
||||
| Any | [pyenv](https://github.com/pyenv/pyenv): `pyenv install 3.13` |
|
||||
|
||||
### uv
|
||||
|
||||
[uv](https://docs.astral.sh/uv/) is the project's fast Python package manager and
|
||||
virtual-environment tool.
|
||||
|
||||
```bash
|
||||
# macOS / Linux
|
||||
curl -LsSf https://astral.sh/uv/install.sh | sh
|
||||
|
||||
# Windows (PowerShell)
|
||||
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
|
||||
|
||||
# Verify
|
||||
uv --version
|
||||
```
|
||||
|
||||
### nox
|
||||
|
||||
[nox](https://nox.thea.codes/) is the project's task runner. All test, lint,
|
||||
type-check, and build commands are executed through `nox` sessions — **never**
|
||||
invoke `behave`, `ruff`, `pyright`, or other tools directly.
|
||||
|
||||
```bash
|
||||
pip install nox
|
||||
# or, using uv:
|
||||
uv tool install nox
|
||||
```
|
||||
|
||||
### Git
|
||||
|
||||
Git 2.30+ is required. Most systems ship a recent enough version.
|
||||
|
||||
```bash
|
||||
git --version # must be 2.30 or later
|
||||
```
|
||||
|
||||
### Commitizen (optional but recommended)
|
||||
|
||||
[Commitizen](https://commitizen.github.io/cz-cli/) guides you through writing
|
||||
[Conventional Changelog](https://github.com/conventional-changelog/conventional-changelog-eslint/blob/master/convention.md)-compliant
|
||||
commit messages interactively. All commits **must** follow this standard.
|
||||
|
||||
```bash
|
||||
npm install -g commitizen@2.8.6 cz-customizable@4.0.0
|
||||
```
|
||||
|
||||
> **Note:** npm is only needed for Commitizen. Node.js is not otherwise required
|
||||
> to develop or run cleveragents-core.
|
||||
|
||||
### pre-commit
|
||||
|
||||
[pre-commit](https://pre-commit.com/) runs automated checks before each commit.
|
||||
It is installed as part of the dev dependencies (see [Getting Started](#2-getting-started))
|
||||
but the CLI must also be available on your PATH.
|
||||
|
||||
```bash
|
||||
pip install pre-commit
|
||||
# or, using uv:
|
||||
uv tool install pre-commit
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 2. Getting Started
|
||||
|
||||
### Clone the Repository
|
||||
|
||||
```bash
|
||||
git clone https://git.cleverthis.com/cleveragents/cleveragents-core.git
|
||||
cd cleveragents-core
|
||||
```
|
||||
|
||||
### Create and Activate a Virtual Environment
|
||||
|
||||
```bash
|
||||
# Create a venv with Python 3.13
|
||||
uv venv --python 3.13
|
||||
|
||||
# Activate (macOS / Linux)
|
||||
source .venv/bin/activate
|
||||
|
||||
# Activate (Windows PowerShell)
|
||||
.venv\Scripts\Activate.ps1
|
||||
```
|
||||
|
||||
### Install All Dependencies
|
||||
|
||||
Install the package in editable mode together with all development and test
|
||||
extras:
|
||||
|
||||
```bash
|
||||
uv pip install -e ".[dev,tests,docs]"
|
||||
```
|
||||
|
||||
This installs:
|
||||
|
||||
- **Runtime** dependencies (typer, pydantic, langchain, etc.)
|
||||
- **Dev** extras: ruff, pyright, behave, pytest, pre-commit, bandit, semgrep,
|
||||
vulture, radon
|
||||
- **Tests** extras: behave, slipcover, asv, robotframework, robotframework-pabot
|
||||
- **Docs** extras: mkdocs, mkdocs-material, mkdocstrings
|
||||
|
||||
### Install pre-commit Hooks
|
||||
|
||||
```bash
|
||||
pre-commit install
|
||||
```
|
||||
|
||||
This installs Git hooks that run linting and formatting checks automatically on
|
||||
`git commit`.
|
||||
|
||||
### Verify the Installation
|
||||
|
||||
Run the full default nox pipeline to confirm everything is working:
|
||||
|
||||
```bash
|
||||
nox
|
||||
```
|
||||
|
||||
This executes (in order): `lint` → `typecheck` → `unit_tests` →
|
||||
`integration_tests` → `coverage_report`. A clean run with all sessions passing
|
||||
means your environment is correctly set up.
|
||||
|
||||
---
|
||||
|
||||
## 3. Development Workflow
|
||||
|
||||
All quality-gate commands are run through `nox`. The `noxfile.py` in the
|
||||
repository root defines every available session.
|
||||
|
||||
### Running Tests
|
||||
|
||||
```bash
|
||||
# Run all default sessions
|
||||
nox
|
||||
|
||||
# Unit tests only (Behave / BDD)
|
||||
nox -s unit_tests
|
||||
|
||||
# Integration tests only (Robot Framework)
|
||||
nox -s integration_tests
|
||||
|
||||
# Coverage report — enforces ≥ 97% threshold
|
||||
nox -s coverage_report
|
||||
|
||||
# Run a single Behave feature file
|
||||
nox -s unit_tests -- features/plan_model.feature
|
||||
|
||||
# Run scenarios with a specific tag
|
||||
nox -s unit_tests -- --tags=@smoke
|
||||
|
||||
# Run a single Robot suite
|
||||
nox -s integration_tests -- --suite robot/plan_persistence_e2e.robot
|
||||
|
||||
# Performance benchmarks (ASV)
|
||||
nox -s benchmark
|
||||
```
|
||||
|
||||
> See [Testing Guide](testing.md) for full details on the testing strategy,
|
||||
> coverage requirements, and test suite structure.
|
||||
|
||||
### Linting
|
||||
|
||||
```bash
|
||||
# Check for lint violations (ruff)
|
||||
nox -s lint
|
||||
|
||||
# Auto-fix safe violations
|
||||
nox -s lint -- --fix
|
||||
|
||||
# Check formatting only (no changes)
|
||||
nox -s format -- --check
|
||||
|
||||
# Apply formatting
|
||||
nox -s format
|
||||
```
|
||||
|
||||
Ruff is configured in `pyproject.toml` under `[tool.ruff]`. The project targets
|
||||
Python 3.13 with an 88-character line length and enforces rules from the `E`,
|
||||
`F`, `W`, `B`, `UP`, `I`, `SIM`, and `RUF` rule sets.
|
||||
|
||||
### Type Checking
|
||||
|
||||
```bash
|
||||
# Run pyright type checker
|
||||
nox -s typecheck
|
||||
```
|
||||
|
||||
Pyright is configured in `pyproject.toml` under `[tool.pyright]`. All source
|
||||
files under `src/` must pass strict type checking. Do not add `# type: ignore`
|
||||
comments without a documented justification.
|
||||
|
||||
### Security Scanning
|
||||
|
||||
```bash
|
||||
# Run bandit + vulture
|
||||
nox -s security_scan
|
||||
|
||||
# Complexity metrics (radon)
|
||||
nox -s complexity
|
||||
```
|
||||
|
||||
### Building Documentation
|
||||
|
||||
```bash
|
||||
# Build the MkDocs site locally
|
||||
nox -s docs
|
||||
|
||||
# Serve docs with live reload at http://127.0.0.1:8000
|
||||
nox -s docs -- serve
|
||||
```
|
||||
|
||||
### Full Quality Gate (pre-PR checklist)
|
||||
|
||||
Before opening a pull request, run the complete quality gate:
|
||||
|
||||
```bash
|
||||
nox
|
||||
```
|
||||
|
||||
All sessions must pass. The CI pipeline enforces the same checks and will block
|
||||
merges on any failure.
|
||||
|
||||
---
|
||||
|
||||
## 4. Commit Guidelines
|
||||
|
||||
### Conventional Changelog Format
|
||||
|
||||
Every commit **must** follow the
|
||||
[Conventional Changelog standard](https://github.com/conventional-changelog/conventional-changelog-eslint/blob/master/convention.md).
|
||||
The format is:
|
||||
|
||||
```
|
||||
<type>(<scope>): <short summary>
|
||||
|
||||
<optional body — wrap at 72 characters>
|
||||
|
||||
<optional footer — ISSUES CLOSED: #N>
|
||||
```
|
||||
|
||||
**Allowed types:**
|
||||
|
||||
| Type | When to use |
|
||||
|------|-------------|
|
||||
| `feat` | New feature or user-visible behaviour |
|
||||
| `fix` | Bug fix |
|
||||
| `docs` | Documentation only |
|
||||
| `refactor` | Code restructuring without behaviour change |
|
||||
| `test` | Adding or updating tests |
|
||||
| `chore` | Build system, tooling, dependency updates |
|
||||
| `perf` | Performance improvement |
|
||||
| `ci` | CI/CD pipeline changes |
|
||||
|
||||
**Example:**
|
||||
|
||||
```
|
||||
fix(concurrency): guard execute_plan with advisory lock
|
||||
|
||||
LockService was implemented but never wired into the plan execution
|
||||
path, leaving execute_plan() unprotected against concurrent calls.
|
||||
|
||||
ISSUES CLOSED: #7989
|
||||
```
|
||||
|
||||
### Using Commitizen (recommended)
|
||||
|
||||
```bash
|
||||
# Instead of `git commit`, use:
|
||||
git cz
|
||||
```
|
||||
|
||||
Commitizen walks you through an interactive prompt to build a correctly
|
||||
formatted commit message. All CleverThis repositories ship a local
|
||||
`cz-customizable` configuration.
|
||||
|
||||
### Atomic Commits
|
||||
|
||||
- **One logical change per commit.** Never bundle unrelated changes.
|
||||
- **Include tests with the change.** Feature + tests are one logical unit.
|
||||
- **Include documentation with the change.** Update relevant docs in the same commit.
|
||||
- **Each commit must build and pass all tests.** The repository must be in a
|
||||
working state at every commit in history.
|
||||
|
||||
See [CONTRIBUTING.md](../../CONTRIBUTING.md) for the full commit quality policy.
|
||||
|
||||
### Branch Naming
|
||||
|
||||
Use the following prefixes:
|
||||
|
||||
| Prefix | Purpose |
|
||||
|--------|---------|
|
||||
| `feat/<issue>-<slug>` | New feature |
|
||||
| `fix/<issue>-<slug>` | Bug fix |
|
||||
| `docs/<slug>` | Documentation |
|
||||
| `refactor/<slug>` | Refactoring |
|
||||
| `chore/<slug>` | Tooling / maintenance |
|
||||
| `test/<slug>` | Test-only changes |
|
||||
|
||||
Examples:
|
||||
|
||||
```
|
||||
feat/issue-1234-plan-correction-rollback
|
||||
fix/7989-lock-service-concurrency
|
||||
docs/developer-setup-guide
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 5. Devcontainer Setup
|
||||
|
||||
The repository ships a [Dev Container](https://containers.dev/) configuration
|
||||
that provides a fully pre-configured development environment with Python 3.13,
|
||||
uv, nox, and all dependencies pre-installed. This is the **recommended** setup
|
||||
for contributors who want zero-friction onboarding.
|
||||
|
||||
### Requirements
|
||||
|
||||
- [Docker Desktop](https://www.docker.com/products/docker-desktop/) (macOS /
|
||||
Windows) or Docker Engine (Linux)
|
||||
- [VS Code](https://code.visualstudio.com/) with the
|
||||
[Dev Containers extension](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers),
|
||||
**or** the [devcontainer CLI](https://github.com/devcontainers/cli)
|
||||
|
||||
### Opening in VS Code
|
||||
|
||||
1. Open the repository folder in VS Code.
|
||||
2. When prompted *"Reopen in Container"*, click **Reopen in Container**.
|
||||
Alternatively: open the Command Palette (`Ctrl+Shift+P` / `Cmd+Shift+P`) and
|
||||
run **Dev Containers: Reopen in Container**.
|
||||
3. VS Code builds the container image and installs all extensions. This takes a
|
||||
few minutes on first run; subsequent opens are fast.
|
||||
4. A terminal inside the container is ready to use. All tools (`uv`, `nox`,
|
||||
`git`, `pre-commit`) are on the PATH.
|
||||
|
||||
### Using the devcontainer CLI
|
||||
|
||||
```bash
|
||||
# Start the container
|
||||
devcontainer up --workspace-folder .
|
||||
|
||||
# Execute a command inside the container
|
||||
devcontainer exec --workspace-folder . nox -s unit_tests
|
||||
```
|
||||
|
||||
### Named Configurations
|
||||
|
||||
The repository may define multiple named devcontainer configurations (e.g.,
|
||||
`default`, `docs-only`). To select one in VS Code, use
|
||||
**Dev Containers: Reopen in Container…** and choose the desired configuration
|
||||
from the list.
|
||||
|
||||
### Environment Variables
|
||||
|
||||
Sensitive configuration (API keys, database URLs) should be placed in a
|
||||
`.env` file at the repository root. The devcontainer automatically mounts
|
||||
this file. **Never commit `.env` to version control.**
|
||||
|
||||
Common variables:
|
||||
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
| `CLEVERAGENTS_DATABASE_URL` | SQLite or PostgreSQL connection string |
|
||||
| `CLEVERAGENTS_HOME` | Override the default data directory |
|
||||
| `CLEVERAGENTS_TESTING_USE_MOCK_AI` | Set to `1` to use mock AI in tests |
|
||||
| `OPENAI_API_KEY` | OpenAI provider key (optional) |
|
||||
| `ANTHROPIC_API_KEY` | Anthropic provider key (optional) |
|
||||
|
||||
---
|
||||
|
||||
## 6. Troubleshooting
|
||||
|
||||
### `python3 --version` shows < 3.13
|
||||
|
||||
Your system Python is too old. Install Python 3.13 via pyenv, your OS package
|
||||
manager, or [python.org](https://www.python.org/downloads/), then recreate the
|
||||
virtual environment:
|
||||
|
||||
```bash
|
||||
uv venv --python 3.13
|
||||
source .venv/bin/activate
|
||||
uv pip install -e ".[dev,tests,docs]"
|
||||
```
|
||||
|
||||
### `nox` command not found
|
||||
|
||||
Install nox into your system Python or via uv tools:
|
||||
|
||||
```bash
|
||||
pip install nox
|
||||
# or
|
||||
uv tool install nox
|
||||
```
|
||||
|
||||
Ensure the tool installation directory is on your `PATH` (uv prints the path
|
||||
after installation).
|
||||
|
||||
### `pre-commit` hooks not running
|
||||
|
||||
Reinstall the hooks:
|
||||
|
||||
```bash
|
||||
pre-commit install --overwrite
|
||||
```
|
||||
|
||||
If hooks still do not run, verify that `.git/hooks/pre-commit` exists and is
|
||||
executable.
|
||||
|
||||
### Import errors after installing dependencies
|
||||
|
||||
Ensure you activated the virtual environment before running any commands:
|
||||
|
||||
```bash
|
||||
source .venv/bin/activate # macOS / Linux
|
||||
.venv\Scripts\Activate.ps1 # Windows
|
||||
```
|
||||
|
||||
Then verify the package is installed in editable mode:
|
||||
|
||||
```bash
|
||||
pip show cleveragents
|
||||
```
|
||||
|
||||
### `nox -s unit_tests` fails with `AmbiguousStep`
|
||||
|
||||
Two Behave step files define the same step pattern. Check the error message for
|
||||
the conflicting step text, then either rename one step or add a unique prefix.
|
||||
See the [Testing Guide](testing.md#unit-tests-behave) for step naming conventions.
|
||||
|
||||
### Coverage below 97%
|
||||
|
||||
Run the HTML coverage report to identify uncovered lines:
|
||||
|
||||
```bash
|
||||
nox -s coverage_report
|
||||
open build/htmlcov/index.html
|
||||
```
|
||||
|
||||
Write Behave scenarios targeting the uncovered code paths. Do **not** lower the
|
||||
threshold or add `# pragma: no cover` without explicit team agreement.
|
||||
|
||||
### `pyright` reports errors in generated or third-party stubs
|
||||
|
||||
Check `pyproject.toml` under `[tool.pyright]` for the `exclude` list. Generated
|
||||
stubs under `docs/reference/contracts/stubs/` are excluded by the ruff
|
||||
configuration and should also be excluded from pyright. Add a targeted
|
||||
`# type: ignore` with a comment only as a last resort.
|
||||
|
||||
### Robot Framework `Resource file does not exist`
|
||||
|
||||
Always use `${CURDIR}/` prefix for `Resource` imports:
|
||||
|
||||
```robot
|
||||
Resource ${CURDIR}/common.resource
|
||||
```
|
||||
|
||||
Bare paths are resolved relative to the working directory, not the test file.
|
||||
|
||||
### Devcontainer fails to start
|
||||
|
||||
1. Ensure Docker is running (`docker info`).
|
||||
2. Try **Dev Containers: Rebuild Container** from the VS Code Command Palette.
|
||||
3. Check the Docker build log in the VS Code Output panel for errors.
|
||||
4. If the issue persists, delete the container image and rebuild:
|
||||
```bash
|
||||
docker system prune -f
|
||||
devcontainer up --workspace-folder . --remove-existing-container
|
||||
```
|
||||
|
||||
### SQLite in-memory tests lose data between steps
|
||||
|
||||
When writing Behave tests with `sqlite:///:memory:`, use a single shared
|
||||
`Session` across all repository calls in a scenario — do not create a new
|
||||
session per call. See the [Testing Guide](testing.md#troubleshooting) for the
|
||||
correct pattern.
|
||||
|
||||
---
|
||||
|
||||
## See Also
|
||||
|
||||
- [Testing Guide](testing.md) — Full testing strategy, coverage requirements,
|
||||
and suite documentation
|
||||
- [CI/CD Pipeline](ci-cd.md) — Forgejo CI job definitions and pipeline stages
|
||||
- [Quality Automation](quality-automation.md) — Automated quality gates and
|
||||
enforcement
|
||||
- [CONTRIBUTING.md](../../CONTRIBUTING.md) — Commit standards, code of conduct,
|
||||
and contribution workflow
|
||||
- [Architecture Decision Records](../adr/index.md) — Key design decisions
|
||||
(ADR-003 DI, ADR-005 Tech Stack, ADR-009 Project Model, etc.)
|
||||
@@ -1,111 +0,0 @@
|
||||
@action
|
||||
@security
|
||||
Feature: Action Schema Environment Variable Security
|
||||
As a security-conscious developer
|
||||
I want to ensure that action YAML files can only interpolate allowlisted environment variables
|
||||
So that sensitive credentials cannot be exfiltrated through malicious action configurations
|
||||
|
||||
Background:
|
||||
Given an action YAML string with only required fields
|
||||
|
||||
# ============================================================
|
||||
# Allowed Environment Variable Interpolation
|
||||
# ============================================================
|
||||
|
||||
Scenario: Interpolate CLEVERAGENTS_ prefixed environment variable
|
||||
Given an environment variable "CLEVERAGENTS_API_KEY" set to "secret-key-123"
|
||||
And an action YAML string with description "${CLEVERAGENTS_API_KEY}"
|
||||
When I load the action YAML
|
||||
Then the action description should be "secret-key-123"
|
||||
|
||||
Scenario: Interpolate multiple CLEVERAGENTS_ prefixed variables
|
||||
Given an environment variable "CLEVERAGENTS_HOST" set to "localhost"
|
||||
And an environment variable "CLEVERAGENTS_PORT" set to "8080"
|
||||
And an action YAML string with description "Connect to ${CLEVERAGENTS_HOST}:${CLEVERAGENTS_PORT}"
|
||||
When I load the action YAML
|
||||
Then the action description should be "Connect to localhost:8080"
|
||||
|
||||
Scenario: Missing CLEVERAGENTS_ variable left as placeholder
|
||||
Given no environment variable "CLEVERAGENTS_MISSING_VAR"
|
||||
And an action YAML string with description "Config: ${CLEVERAGENTS_MISSING_VAR}"
|
||||
When I load the action YAML
|
||||
Then the action description should be "Config: ${CLEVERAGENTS_MISSING_VAR}"
|
||||
|
||||
# ============================================================
|
||||
# Disallowed Environment Variable Protection
|
||||
# ============================================================
|
||||
|
||||
Scenario: Disallow AWS_SECRET_ACCESS_KEY interpolation
|
||||
Given an environment variable "AWS_SECRET_ACCESS_KEY" set to "aws-secret-123"
|
||||
And an action YAML string with description "Secret: ${AWS_SECRET_ACCESS_KEY}"
|
||||
When I load the action YAML
|
||||
Then the action description should be "Secret: ${AWS_SECRET_ACCESS_KEY}"
|
||||
|
||||
Scenario: Disallow DATABASE_PASSWORD interpolation
|
||||
Given an environment variable "DATABASE_PASSWORD" set "db-password-456"
|
||||
And an action YAML string with description "DB: ${DATABASE_PASSWORD}"
|
||||
When I load the action YAML
|
||||
Then the action description should be "DB: ${DATABASE_PASSWORD}"
|
||||
|
||||
Scenario: Disallow API_KEY interpolation
|
||||
Given an environment variable "API_KEY" set to "api-key-789"
|
||||
And an action YAML string with description "API: ${API_KEY}"
|
||||
When I load the action YAML
|
||||
Then the action description should be "API: ${API_KEY}"
|
||||
|
||||
Scenario: Disallow HOME interpolation
|
||||
Given an environment variable "HOME" set to "/home/user"
|
||||
And an action YAML string with description "Home: ${HOME}"
|
||||
When I load the action YAML
|
||||
Then the action description should be "Home: ${HOME}"
|
||||
|
||||
Scenario: Disallow PATH interpolation
|
||||
Given an environment variable "PATH" set to "/usr/bin:/bin"
|
||||
And an action YAML string with description "Path: ${PATH}"
|
||||
When I load the action YAML
|
||||
Then the action description should be "Path: ${PATH}"
|
||||
|
||||
# ============================================================
|
||||
# Mixed Allowed and Disallowed Variables
|
||||
# ============================================================
|
||||
|
||||
Scenario: Mix allowed and disallowed variables
|
||||
Given an environment variable "CLEVERAGENTS_CONFIG" set to "allowed-config"
|
||||
And an environment variable "SECRET_TOKEN" set to "secret-token-xyz"
|
||||
And an action YAML string with description "Config: ${CLEVERAGENTS_CONFIG}, Token: ${SECRET_TOKEN}"
|
||||
When I load the action YAML
|
||||
Then the action description should be "Config: allowed-config, Token: ${SECRET_TOKEN}"
|
||||
|
||||
# ============================================================
|
||||
# Edge Cases
|
||||
# ============================================================
|
||||
|
||||
Scenario: Empty prefix check (CLEVERAGENTS_ variable is interpolated)
|
||||
Given an environment variable "CLEVERAGENTS_" set to "empty-prefix"
|
||||
And an action YAML string with description "Var: ${CLEVERAGENTS_}"
|
||||
When I load the action YAML
|
||||
Then the action description should be "Var: empty-prefix"
|
||||
|
||||
Scenario: Case sensitivity - lowercase not allowed
|
||||
Given an environment variable "cleveragents_api_key" set to "lowercase-key"
|
||||
And an action YAML string with description "Key: ${cleveragents_api_key}"
|
||||
When I load the action YAML
|
||||
Then the action description should be "Key: ${cleveragents_api_key}"
|
||||
|
||||
Scenario: Interpolation in nested fields
|
||||
Given an environment variable "CLEVERAGENTS_ACTOR" set to "openai/gpt-4"
|
||||
And an action YAML string with strategy_actor "${CLEVERAGENTS_ACTOR}"
|
||||
When I load the action YAML
|
||||
Then the action strategy_actor should be "openai/gpt-4"
|
||||
|
||||
Scenario: Interpolation in list fields
|
||||
Given an environment variable "CLEVERAGENTS_INVARIANT" set to "No secrets in code"
|
||||
And an action YAML string with invariants containing "${CLEVERAGENTS_INVARIANT}"
|
||||
When I load the action YAML
|
||||
Then the action invariants should contain "No secrets in code"
|
||||
|
||||
Scenario: Disallowed variable in list fields
|
||||
Given an environment variable "SECRET_INVARIANT" set to "Secret invariant"
|
||||
And an action YAML string with invariants containing "${SECRET_INVARIANT}"
|
||||
When I load the action YAML
|
||||
Then the action invariants should contain "${SECRET_INVARIANT}"
|
||||
@@ -1,104 +0,0 @@
|
||||
"""Step definitions for action schema environment variable security tests.
|
||||
|
||||
Tests for features/action_schema_env_var_security.feature — validates that
|
||||
the ActionConfigSchema only allows interpolation of CLEVERAGENTS_ prefixed
|
||||
environment variables, preventing exfiltration of sensitive credentials.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
from behave import given, when # type: ignore[import-untyped]
|
||||
from behave.runner import Context # type: ignore[import-untyped]
|
||||
|
||||
from cleveragents.action.schema import ActionConfigSchema
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# Given steps (unique to this feature)
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@given('an environment variable "{var_name}" set to "{var_value}"')
|
||||
def step_given_env_var_set(context: Context, var_name: str, var_value: str) -> None:
|
||||
"""Set an environment variable for testing."""
|
||||
os.environ[var_name] = var_value
|
||||
# Store for cleanup
|
||||
if not hasattr(context, "env_vars_to_clean"):
|
||||
context.env_vars_to_clean = []
|
||||
context.env_vars_to_clean.append(var_name)
|
||||
|
||||
|
||||
@given('an environment variable "{var_name}" set "{var_value}"')
|
||||
def step_given_env_var_set_alt(context: Context, var_name: str, var_value: str) -> None:
|
||||
"""Set an environment variable for testing (alternative syntax)."""
|
||||
os.environ[var_name] = var_value
|
||||
# Store for cleanup
|
||||
if not hasattr(context, "env_vars_to_clean"):
|
||||
context.env_vars_to_clean = []
|
||||
context.env_vars_to_clean.append(var_name)
|
||||
|
||||
|
||||
@given('no environment variable "{var_name}"')
|
||||
def step_given_no_env_var(context: Context, var_name: str) -> None:
|
||||
"""Ensure an environment variable is not set."""
|
||||
if var_name in os.environ:
|
||||
del os.environ[var_name]
|
||||
|
||||
|
||||
@given('an action YAML string with description "{description}"')
|
||||
def step_given_yaml_with_description(context: Context, description: str) -> None:
|
||||
"""Provide YAML with a custom description value."""
|
||||
_MINIMAL_YAML = """\
|
||||
name: local/simple-action
|
||||
description: A simple action for testing
|
||||
strategy_actor: openai/gpt-4
|
||||
execution_actor: openai/gpt-4
|
||||
definition_of_done: All tests pass
|
||||
"""
|
||||
context.action_yaml_string = _MINIMAL_YAML.replace(
|
||||
"A simple action for testing", description
|
||||
)
|
||||
|
||||
|
||||
@given('an action YAML string with strategy_actor "{actor}"')
|
||||
def step_given_yaml_with_strategy_actor(context: Context, actor: str) -> None:
|
||||
"""Provide YAML with a custom strategy_actor value."""
|
||||
_MINIMAL_YAML = """\
|
||||
name: local/simple-action
|
||||
description: A simple action for testing
|
||||
strategy_actor: openai/gpt-4
|
||||
execution_actor: openai/gpt-4
|
||||
definition_of_done: All tests pass
|
||||
"""
|
||||
context.action_yaml_string = _MINIMAL_YAML.replace("openai/gpt-4", actor, 1)
|
||||
|
||||
|
||||
@given('an action YAML string with invariants containing "{invariant}"')
|
||||
def step_given_yaml_with_invariants(context: Context, invariant: str) -> None:
|
||||
"""Provide YAML with invariants containing a specific value."""
|
||||
_MINIMAL_YAML = """\
|
||||
name: local/simple-action
|
||||
description: A simple action for testing
|
||||
strategy_actor: openai/gpt-4
|
||||
execution_actor: openai/gpt-4
|
||||
definition_of_done: All tests pass
|
||||
"""
|
||||
yaml_with_invariants = _MINIMAL_YAML + f"\ninvariants:\n - {invariant}\n"
|
||||
context.action_yaml_string = yaml_with_invariants
|
||||
|
||||
|
||||
# ────────────────────────────────────────────────────────────
|
||||
# When steps (unique to this feature)
|
||||
# ────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@when("I load the action YAML")
|
||||
def step_when_load_action_yaml(context: Context) -> None:
|
||||
"""Load and validate the action YAML."""
|
||||
try:
|
||||
context.action_config = ActionConfigSchema.from_yaml(context.action_yaml_string)
|
||||
context.load_error = None
|
||||
except Exception as e:
|
||||
context.action_config = None
|
||||
context.load_error = e
|
||||
@@ -30,6 +30,7 @@ nav:
|
||||
- ACMS Context Hydration: modules/context-hydration.md
|
||||
- Git Worktree Sandbox: modules/git-worktree-sandbox.md
|
||||
- Development:
|
||||
- Setup Guide: development/setup.md
|
||||
- Agent System Specification: development/agent-system-specification.md
|
||||
- CI/CD Pipeline: development/ci-cd.md
|
||||
- Quality Automation: development/quality-automation.md
|
||||
|
||||
@@ -45,9 +45,6 @@ NAMESPACED_NAME_RE = re.compile(r"^[a-z0-9][a-z0-9_-]*/[a-z0-9][a-z0-9_-]*$")
|
||||
#: Pattern for ``${VAR}`` environment variable references.
|
||||
_ENV_VAR_RE = re.compile(r"\$\{([A-Za-z_][A-Za-z0-9_]*)\}")
|
||||
|
||||
#: Allowed prefix for environment variable interpolation (security restriction).
|
||||
_ALLOWED_ENV_VAR_PREFIX = "CLEVERAGENTS_"
|
||||
|
||||
#: Supported action states (draft excluded from YAML configs).
|
||||
_VALID_STATES = frozenset({"available", "archived"})
|
||||
|
||||
@@ -451,10 +448,6 @@ def _interpolate_env_vars(data: dict[str, Any]) -> dict[str, Any]:
|
||||
|
||||
Only string values are interpolated. Missing environment variables
|
||||
are left as-is (no error) to allow deferred resolution.
|
||||
|
||||
Security: Only environment variables with the ``CLEVERAGENTS_`` prefix
|
||||
are eligible for interpolation. Attempts to reference other variables
|
||||
are left as literal ``${VAR}`` placeholders.
|
||||
"""
|
||||
result: dict[str, Any] = {}
|
||||
for key, value in data.items():
|
||||
@@ -481,13 +474,6 @@ def _interpolate_env_vars(data: dict[str, Any]) -> dict[str, Any]:
|
||||
|
||||
|
||||
def _env_replacer(match: re.Match[str]) -> str:
|
||||
"""Replace a single ``${VAR}`` match with its env value.
|
||||
|
||||
Security: Only allows interpolation of environment variables with the
|
||||
``CLEVERAGENTS_`` prefix. Other variables are left as literal placeholders.
|
||||
"""
|
||||
"""Replace a single ``${VAR}`` match with its env value."""
|
||||
var_name = match.group(1)
|
||||
if var_name.startswith(_ALLOWED_ENV_VAR_PREFIX):
|
||||
return os.environ.get(var_name, match.group(0))
|
||||
# Return the original placeholder for disallowed variables
|
||||
return match.group(0)
|
||||
return os.environ.get(var_name, match.group(0))
|
||||
|
||||
Reference in New Issue
Block a user