HAL9000
dde71cf8e2
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 1m12s
CI / quality (pull_request) Successful in 1m24s
CI / push-validation (pull_request) Successful in 33s
CI / helm (pull_request) Successful in 34s
CI / typecheck (pull_request) Successful in 1m33s
CI / build (pull_request) Successful in 1m13s
CI / benchmark-regression (pull_request) Failing after 37s
CI / security (pull_request) Successful in 2m25s
CI / e2e_tests (pull_request) Successful in 4m31s
CI / unit_tests (pull_request) Successful in 6m25s
CI / integration_tests (pull_request) Successful in 6m24s
CI / docker (pull_request) Successful in 1m36s
CI / coverage (pull_request) Successful in 13m7s
CI / coverage (push) Blocked by required conditions
CI / docker (push) Blocked by required conditions
CI / status-check (push) Blocked by required conditions
CI / benchmark-publish (push) Waiting to run
CI / benchmark-regression (push) Waiting to run
CI / status-check (pull_request) Successful in 5s
CI / push-validation (push) Successful in 31s
CI / quality (push) Successful in 1m22s
CI / lint (push) Successful in 1m28s
CI / helm (push) Successful in 43s
CI / build (push) Successful in 1m0s
CI / security (push) Successful in 1m41s
CI / typecheck (push) Successful in 2m5s
CI / e2e_tests (push) Successful in 4m13s
CI / integration_tests (push) Successful in 4m37s
CI / unit_tests (push) Has been cancelled
Split docs/advanced-concepts.md (554 lines) into four sub-documents under docs/advanced-concepts/ and docs/tui.md (634 lines) into four sub-documents under docs/tui/, each under the 500-line limit per CONTRIBUTING.md. Advanced Concepts sub-documents: - docs/advanced-concepts/index.md: Overview, context strategies, LLM backends - docs/advanced-concepts/resource-types.md: Resource types, A2A rename - docs/advanced-concepts/container-tools.md: Container tools, scope chain, budgets, safety - docs/advanced-concepts/e2e-tests-and-plugins.md: E2E tests, code review, plugins TUI sub-documents: - docs/tui/index.md: Overview, getting started, main screen layout - docs/tui/sidebar-and-personas.md: Sidebar states, persona system - docs/tui/input-and-sessions.md: Reference/command input, session management - docs/tui/configuration-and-integration.md: Config, key bindings, theme, integration Updated mkdocs.yml navigation to reflect new sub-document structure. Updated CONTRIBUTORS.md with contribution entry for PR #9903. ISSUES CLOSED: #10533
139 lines
5.4 KiB
Markdown
139 lines
5.4 KiB
Markdown
# Advanced Concepts: E2E Tests, Code Review Examples & Plugin Architecture (v3.6.0)
|
|
|
|
> **Milestone:** v3.6.0 — M7: Advanced Concepts & Deferred Features
|
|
> **Parent:** [Advanced Concepts Overview](index.md)
|
|
|
|
---
|
|
|
|
## End-to-End Workflow Specification Tests
|
|
|
|
v3.6.0 introduces a new category of tests: **E2E workflow specification tests**. These tests
|
|
verify complete multi-step workflows from the user's perspective, exercising the full stack
|
|
from CLI input through plan execution to file output.
|
|
|
|
### Test Categories
|
|
|
|
| Category | Description | Framework |
|
|
|----------|-------------|-----------|
|
|
| `e2e/workflow` | Full plan lifecycle: create to strategize to execute to apply | Robot Framework |
|
|
| `e2e/multi-project` | Workflows spanning multiple projects | Robot Framework |
|
|
| `e2e/provider` | Provider-specific behavior (rate limits, fallback, cost tracking) | Behave |
|
|
| `e2e/safety` | Safety profile enforcement under various conditions | Behave |
|
|
| `e2e/budget` | Budget enforcement and plan pause/resume | Behave |
|
|
|
|
### Running E2E Tests
|
|
|
|
```bash
|
|
nox -s e2e # Run all E2E tests
|
|
nox -s e2e -- --tags workflow # Run only workflow tests
|
|
nox -s e2e -- --tags provider # Run only provider tests
|
|
```
|
|
|
|
E2E tests require a configured LLM provider. Set `E2E_PROVIDER=anthropic` (or another
|
|
supported provider) in your environment before running.
|
|
|
|
---
|
|
|
|
## Code Review Tool Examples
|
|
|
|
v3.6.0 ships a set of example actors and skills specifically designed for code review
|
|
workflows. These serve as both functional tools and reference implementations for the actor
|
|
and skill systems.
|
|
|
|
### Included Examples
|
|
|
|
| Example | Location | Description |
|
|
|---------|----------|-------------|
|
|
| `code-reviewer` | `examples/actors/code-reviewer.yaml` | Full code review actor with diff analysis, style checking, and security scanning |
|
|
| `pr-summarizer` | `examples/actors/pr-summarizer.yaml` | Summarizes pull request changes for review |
|
|
| `test-generator` | `examples/actors/test-generator.yaml` | Generates BDD test scenarios from source code |
|
|
| `doc-writer` | `examples/actors/doc-writer.yaml` | Generates and updates documentation from code |
|
|
| `security-scanner` | `examples/skills/security-scanner.yaml` | Skill that runs security analysis tools |
|
|
| `style-checker` | `examples/skills/style-checker.yaml` | Skill that runs linting and formatting checks |
|
|
|
|
### Using the Code Reviewer
|
|
|
|
```bash
|
|
# Register the example actor
|
|
agents actor register examples/actors/code-reviewer.yaml
|
|
|
|
# Create a code review plan
|
|
agents plan use code-review \
|
|
--project local/my-project \
|
|
--arg diff_source=git \
|
|
--arg base_branch=main
|
|
```
|
|
|
|
---
|
|
|
|
## Plugin Architecture Extensions
|
|
|
|
v3.6.0 formalizes and extends the plugin architecture, making it easier to add custom
|
|
resource types, tool executors, scope resolvers, and LLM providers.
|
|
|
|
### Plugin Entry Points
|
|
|
|
Plugins are registered via Python package entry points:
|
|
|
|
```toml
|
|
# pyproject.toml
|
|
[project.entry-points."cleveragents.plugins"]
|
|
my-plugin = "my_package.plugin:MyPlugin"
|
|
|
|
[project.entry-points."cleveragents.resource_types"]
|
|
my-resource = "my_package.resources:MyResourceType"
|
|
|
|
[project.entry-points."cleveragents.tool_executors"]
|
|
my-executor = "my_package.executors:MyToolExecutor"
|
|
|
|
[project.entry-points."cleveragents.scope_resolvers"]
|
|
my-resolver = "my_package.resolvers:MyScopeResolver"
|
|
|
|
[project.entry-points."cleveragents.providers"]
|
|
my-provider = "my_package.providers:MyLLMProvider"
|
|
```
|
|
|
|
### Plugin Security
|
|
|
|
The `PluginLoader` (hardened in v3.5.0) enforces:
|
|
|
|
- **Module allowlist** — only modules from registered, trusted packages may be loaded
|
|
- **Entry point validation** — entry point targets are parsed and validated before import
|
|
- **Sandbox isolation** — plugins run in a restricted execution environment
|
|
- **Capability declaration** — plugins must declare their required capabilities at registration
|
|
|
|
See ADR-037 (Tool Reachability & Access Projection) for the full security model.
|
|
|
|
### Plugin Development Guide
|
|
|
|
A plugin development guide is available at `docs/development/plugin-development.md` (added
|
|
in v3.6.0). It covers:
|
|
|
|
1. Creating a minimal plugin package
|
|
2. Registering resource types, tool executors, and scope resolvers
|
|
3. Testing plugins with the CleverAgents test harness
|
|
4. Publishing plugins to PyPI
|
|
|
|
---
|
|
|
|
## Related ADRs
|
|
|
|
The following Architecture Decision Records are directly relevant to v3.6.0 features:
|
|
|
|
| ADR | Title | Relevance |
|
|
|-----|-------|-----------|
|
|
| [ADR-008](../adr/ADR-008-resource-system.md) | Resource System | Foundation for additional resource types |
|
|
| [ADR-036](../adr/ADR-036-resource-dag-operational-semantics.md) | Resource DAG Operational Semantics | DAG model for new resource types |
|
|
| [ADR-039](../adr/ADR-039-container-resource-types.md) | Container Resource Types | Container tool execution |
|
|
| [ADR-041](../adr/ADR-041-safety-profile-extraction.md) | Safety Profile Extraction | Safety profiles and cost budgets |
|
|
| [ADR-042](../adr/ADR-042-resource-type-inheritance.md) | Resource Type Inheritance | Custom resource type extensions |
|
|
| [ADR-043](../adr/ADR-043-devcontainer-integration.md) | Devcontainer Integration | Devcontainer tool execution |
|
|
| [ADR-047](../adr/ADR-047-acp-standard-adoption.md) | A2A Standard Adoption | ACP to A2A rename |
|
|
|
|
---
|
|
|
|
*Documentation for v3.6.0 — IN PROGRESS. Features described here reflect the planned scope
|
|
as of 2026-04-15. Some features may be adjusted as implementation progresses.*
|
|
|
|
*Generated by [AUTO-DOCS-5] — CleverAgents Documentation Bot*
|