cleveragents-core/docs/advanced-concepts/e2e-tests-and-plugins.md

Advanced Concepts: E2E Tests, Code Review Examples & Plugin Architecture (v3.6.0)

Milestone: v3.6.0 — M7: Advanced Concepts & Deferred Features Parent: Advanced Concepts Overview

---

End-to-End Workflow Specification Tests

v3.6.0 introduces a new category of tests: E2E workflow specification tests. These tests verify complete multi-step workflows from the user's perspective, exercising the full stack from CLI input through plan execution to file output.

Test Categories

Category	Description	Framework
e2e/workflow	Full plan lifecycle: create to strategize to execute to apply	Robot Framework
e2e/multi-project	Workflows spanning multiple projects	Robot Framework
e2e/provider	Provider-specific behavior (rate limits, fallback, cost tracking)	Behave
e2e/safety	Safety profile enforcement under various conditions	Behave
e2e/budget	Budget enforcement and plan pause/resume	Behave

Running E2E Tests

nox -s e2e                          # Run all E2E tests
nox -s e2e -- --tags workflow       # Run only workflow tests
nox -s e2e -- --tags provider       # Run only provider tests

E2E tests require a configured LLM provider. Set E2E_PROVIDER=anthropic (or another supported provider) in your environment before running.

---

Code Review Tool Examples

v3.6.0 ships a set of example actors and skills specifically designed for code review workflows. These serve as both functional tools and reference implementations for the actor and skill systems.

Included Examples

Example	Location	Description
code-reviewer	examples/actors/code-reviewer.yaml	Full code review actor with diff analysis, style checking, and security scanning
pr-summarizer	examples/actors/pr-summarizer.yaml	Summarizes pull request changes for review
test-generator	examples/actors/test-generator.yaml	Generates BDD test scenarios from source code
doc-writer	examples/actors/doc-writer.yaml	Generates and updates documentation from code
security-scanner	examples/skills/security-scanner.yaml	Skill that runs security analysis tools
style-checker	examples/skills/style-checker.yaml	Skill that runs linting and formatting checks

Using the Code Reviewer

# Register the example actor
agents actor register examples/actors/code-reviewer.yaml

# Create a code review plan
agents plan use code-review \
  --project local/my-project \
  --arg diff_source=git \
  --arg base_branch=main

---

Plugin Architecture Extensions

v3.6.0 formalizes and extends the plugin architecture, making it easier to add custom resource types, tool executors, scope resolvers, and LLM providers.

Plugin Entry Points

Plugins are registered via Python package entry points:

# pyproject.toml
[project.entry-points."cleveragents.plugins"]
my-plugin = "my_package.plugin:MyPlugin"

[project.entry-points."cleveragents.resource_types"]
my-resource = "my_package.resources:MyResourceType"

[project.entry-points."cleveragents.tool_executors"]
my-executor = "my_package.executors:MyToolExecutor"

[project.entry-points."cleveragents.scope_resolvers"]
my-resolver = "my_package.resolvers:MyScopeResolver"

[project.entry-points."cleveragents.providers"]
my-provider = "my_package.providers:MyLLMProvider"

Plugin Security

The PluginLoader (hardened in v3.5.0) enforces:

• Module allowlist — only modules from registered, trusted packages may be loaded
• Entry point validation — entry point targets are parsed and validated before import
• Sandbox isolation — plugins run in a restricted execution environment
• Capability declaration — plugins must declare their required capabilities at registration

See ADR-037 (Tool Reachability & Access Projection) for the full security model.

Plugin Development Guide

A plugin development guide is available at docs/development/plugin-development.md (added in v3.6.0). It covers:

1. Creating a minimal plugin package
2. Registering resource types, tool executors, and scope resolvers
3. Testing plugins with the CleverAgents test harness
4. Publishing plugins to PyPI

---

Related ADRs

The following Architecture Decision Records are directly relevant to v3.6.0 features:

ADR	Title	Relevance
ADR-008	Resource System	Foundation for additional resource types
ADR-036	Resource DAG Operational Semantics	DAG model for new resource types
ADR-039	Container Resource Types	Container tool execution
ADR-041	Safety Profile Extraction	Safety profiles and cost budgets
ADR-042	Resource Type Inheritance	Custom resource type extensions
ADR-043	Devcontainer Integration	Devcontainer tool execution
ADR-047	A2A Standard Adoption	ACP to A2A rename

---

Documentation for v3.6.0 — IN PROGRESS. Features described here reflect the planned scope as of 2026-04-15. Some features may be adjusted as implementation progresses.

Generated by [AUTO-DOCS-5] — CleverAgents Documentation Bot