diff --git a/docs/api/acms.md b/docs/api/acms.md new file mode 100644 index 000000000..7683f484e --- /dev/null +++ b/docs/api/acms.md @@ -0,0 +1,331 @@ +# `cleveragents.acms` — Advanced Context Management System + +The `acms` package implements the Advanced Context Management System (ACMS), +providing UKO (Universal Knowledge Ontology) vocabulary extensions for +context assembly. It defines Layer 2 paradigm vocabularies (Object-Oriented, +Functional, Procedural) and Layer 3 technology-specific vocabulary extensions +for Python, TypeScript, Rust, and Java. + +See [ADR-014](../adr/ADR-014-context-management-acms.md) for the design +rationale and [`docs/reference/acms.md`](../reference/acms.md) for the full +ACMS reference. + +--- + +## Overview + +The ACMS vocabulary system is organized in four layers: + +| Layer | Prefix | Description | +|-------|--------|-------------| +| 0 | `uko:` | Universal foundation — base classes and properties | +| 1 | `uko-code:` | Code-level abstractions | +| 2 | `uko-oo:`, `uko-func:`, `uko-proc:` | Paradigm specializations | +| 3 | `uko-py:`, `uko-ts:`, `uko-rs:`, `uko-java:` | Technology-specific extensions | + +```python +from cleveragents.acms import ( + # Layer 2 paradigm vocabularies + get_oo_vocabulary, + get_func_vocabulary, + get_proc_vocabulary, + # Layer 3 technology vocabularies + PYTHON_VOCABULARY, + TYPESCRIPT_VOCABULARY, + RUST_VOCABULARY, + JAVA_VOCABULARY, + # Detail level resolution + resolve_detail_level, + build_detail_level_map, + build_effective_map, + # Registry + VocabularyRegistry, +) + +# Resolve a detail level for a Python class +oo_map = get_oo_vocabulary().detail_level_map +py_map = PYTHON_VOCABULARY.detail_level_map +effective = build_effective_map(py_map, oo_map) +level = resolve_detail_level("PythonClass", effective) +``` + +--- + +## Base Vocabulary Types + +### `UKOVocabulary` + +Abstract base class for all UKO vocabulary definitions. + +| Attribute | Type | Description | +|-----------|------|-------------| +| `namespace` | `str` | Namespace prefix (e.g. `uko-py:`). | +| `iri` | `str` | Full IRI for the namespace. | +| `layer` | `int` | Ontology layer (0–3). | +| `classes` | `list[UKOClass]` | Classes defined in this vocabulary. | +| `properties` | `list[UKOProperty]` | Properties defined in this vocabulary. | +| `detail_level_map` | `DetailLevelMap` | Maps class names to detail levels. | + +### `UKOClass` + +Represents a class in the UKO ontology. + +| Attribute | Type | Description | +|-----------|------|-------------| +| `name` | `str` | Class name (e.g. `PythonClass`). | +| `iri` | `str` | Full IRI for the class. | +| `parent` | `str \| None` | Parent class IRI (for inheritance). | +| `description` | `str` | Human-readable description. | + +### `UKOProperty` + +Represents a property in the UKO ontology. + +| Attribute | Type | Description | +|-----------|------|-------------| +| `name` | `str` | Property name. | +| `iri` | `str` | Full IRI for the property. | +| `domain` | `str` | Domain class IRI. | +| `range` | `str` | Range class or datatype IRI. | +| `description` | `str` | Human-readable description. | + +### `Layer2Dependency` + +Declares a dependency from a Layer 3 vocabulary on a Layer 2 vocabulary. + +```python +Layer2Dependency( + layer2_namespace="uko-oo:", + layer2_iri="https://cleveragents.ai/ontology/uko/oo#", +) +``` + +### `ProvenanceInfo` + +Provenance metadata attached to vocabulary definitions. + +| Attribute | Type | Description | +|-----------|------|-------------| +| `source` | `str` | Source document or specification reference. | +| `version` | `str` | Version when this vocabulary was introduced. | + +--- + +## Layer 2 — Paradigm Vocabularies + +### `ParadigmVocabulary` + +Extends `UKOVocabulary` for Layer 2 paradigm specializations. + +### `VocabularyClass` + +Extends `UKOClass` with paradigm-specific metadata. + +### `VocabularyProperty` + +Extends `UKOProperty` with paradigm-specific metadata. + +### Factory Functions + +```python +from cleveragents.acms import get_oo_vocabulary, get_func_vocabulary, get_proc_vocabulary + +oo = get_oo_vocabulary() # Object-Oriented paradigm (uko-oo:) +func = get_func_vocabulary() # Functional paradigm (uko-func:) +proc = get_proc_vocabulary() # Procedural paradigm (uko-proc:) +``` + +--- + +## Layer 3 — Technology Vocabularies + +### Python (`uko-py:`) + +```python +from cleveragents.acms import ( + PYTHON_VOCABULARY, + PYTHON_DETAIL_LEVELS, + PythonClass, + PythonFunction, + PythonModule, + PythonDecorator, + PythonTypeStub, +) +``` + +| Class | Description | +|-------|-------------| +| `PythonClass` | Python class definition (`class Foo:`). | +| `PythonFunction` | Python function or method (`def foo():`). | +| `PythonModule` | Python module (`.py` file). | +| `PythonDecorator` | Python decorator (`@decorator`). | +| `PythonTypeStub` | Python type stub (`.pyi` file). | + +### TypeScript (`uko-ts:`) + +```python +from cleveragents.acms import ( + TYPESCRIPT_VOCABULARY, + TYPESCRIPT_DETAIL_LEVELS, + TypeScriptClass, + TypeScriptFunction, + TypeScriptInterface, + TypeScriptModule, +) +``` + +| Class | Description | +|-------|-------------| +| `TypeScriptClass` | TypeScript class definition. | +| `TypeScriptFunction` | TypeScript function or method. | +| `TypeScriptInterface` | TypeScript interface declaration. | +| `TypeScriptModule` | TypeScript module (`.ts` / `.tsx` file). | + +### Rust (`uko-rs:`) + +```python +from cleveragents.acms import ( + RUST_VOCABULARY, + RUST_DETAIL_LEVELS, + RustStruct, + RustTrait, + RustImpl, + RustFunction, + RustDeriveAttribute, +) +``` + +| Class | Description | +|-------|-------------| +| `RustStruct` | Rust struct definition. | +| `RustTrait` | Rust trait definition. | +| `RustImpl` | Rust `impl` block. | +| `RustFunction` | Rust function or method. | +| `RustDeriveAttribute` | Rust `#[derive(...)]` attribute. | + +### Java (`uko-java:`) + +```python +from cleveragents.acms import ( + JAVA_VOCABULARY, + JAVA_DETAIL_LEVELS, + JavaClass, + JavaInterface, + JavaMethod, + JavaAnnotation, + JavaCheckedException, +) +``` + +| Class | Description | +|-------|-------------| +| `JavaClass` | Java class definition. | +| `JavaInterface` | Java interface declaration. | +| `JavaMethod` | Java method definition. | +| `JavaAnnotation` | Java annotation type. | +| `JavaCheckedException` | Java checked exception class. | + +--- + +## Detail Level System + +Detail levels control how much information is included when a resource is +indexed into the knowledge graph. The system uses a hierarchical inheritance +mechanism: Layer 3 maps inherit from Layer 2, which inherit from Layer 1. + +### Pre-built Maps + +```python +from cleveragents.acms import ( + CODE_DETAIL_LEVEL_MAP, # Layer 1 — base code detail levels + OO_DETAIL_LEVEL_MAP, # Layer 2 — OO paradigm detail levels + FUNC_DETAIL_LEVEL_MAP, # Layer 2 — Functional paradigm detail levels + PROC_DETAIL_LEVEL_MAP, # Layer 2 — Procedural paradigm detail levels + PYTHON_DETAIL_LEVELS, # Layer 3 — Python detail levels + TYPESCRIPT_DETAIL_LEVELS,# Layer 3 — TypeScript detail levels + RUST_DETAIL_LEVELS, # Layer 3 — Rust detail levels + JAVA_DETAIL_LEVELS, # Layer 3 — Java detail levels +) +``` + +### `resolve_detail_level` + +```python +level = resolve_detail_level(class_name: str, detail_level_map: dict) -> int +``` + +Resolve the detail level for a named class using the provided map. +Returns the level integer (higher = more detail). + +### `build_detail_level_map` + +```python +map = build_detail_level_map(vocabulary: UKOVocabulary) -> dict[str, int] +``` + +Build a detail level map from a vocabulary's class definitions. + +### `build_effective_map` + +```python +effective = build_effective_map(child_map: dict, parent_map: dict) -> dict[str, int] +``` + +Merge a child map with a parent map, with child entries taking precedence. +Used to resolve Layer 3 detail levels against their Layer 2 parents. + +### `DetailLevelMapBuilder` + +Fluent builder for constructing custom detail level maps: + +```python +from cleveragents.acms import DetailLevelMapBuilder + +custom_map = ( + DetailLevelMapBuilder() + .add("MyClass", level=3) + .add("MyFunction", level=2) + .build() +) +``` + +--- + +## `VocabularyRegistry` + +Global registry for all UKO vocabulary instances. + +```python +from cleveragents.acms import VocabularyRegistry + +registry = VocabularyRegistry() +registry.register(PYTHON_VOCABULARY) +vocab = registry.get("uko-py:") +all_vocabs = registry.list_all() +``` + +### Methods + +| Method | Signature | Description | +|--------|-----------|-------------| +| `register` | `(vocab: UKOVocabulary) → None` | Register a vocabulary. Raises `DuplicateVocabularyError` if already registered. | +| `get` | `(namespace: str) → UKOVocabulary \| None` | Look up by namespace prefix. | +| `list_all` | `() → list[UKOVocabulary]` | Return all registered vocabularies. | + +### `DuplicateVocabularyError` + +Raised when attempting to register a vocabulary with a namespace that is +already registered. + +--- + +## Related Documentation + +- [ADR-014 Context Management (ACMS)](../adr/ADR-014-context-management-acms.md) +- [ACMS Reference](../reference/acms.md) +- [ACMS Context Strategies](../reference/context_strategies.md) +- [UKO Ontology](../reference/uko.md) +- [UKO Runtime](../reference/uko_runtime.md) +- [UKO Layer 2 Paradigm Vocabularies](../reference/uko_layer2_paradigm_vocabularies.md) +- [UKO Layer 3 Technology Vocabularies](../reference/uko_layer3_technology_vocabularies.md) +- [UKO Provenance Tracking](../modules/uko-provenance.md) diff --git a/docs/api/index.md b/docs/api/index.md index 4eee7341e..855fdd04b 100644 --- a/docs/api/index.md +++ b/docs/api/index.md @@ -18,6 +18,8 @@ classes, functions, and exceptions with signatures and usage examples. | [`cleveragents.config`](config.md) | Application settings, logging, metrics, and security scanning | | [`cleveragents.providers`](providers.md) | AI provider registry — discovery, selection, LLM factory, and capability metadata | | [`cleveragents.tui`](tui.md) | Interactive Terminal UI — app, persona system, input routing, slash commands, session export/import | +| [`cleveragents.lsp`](lsp.md) | Language Server Protocol (LSP) integration — registry, runtime, lifecycle manager, and tool adapter | +| [`cleveragents.acms`](acms.md) | Advanced Context Management System — UKO vocabulary extensions for Python, TypeScript, Rust, and Java | > **Note:** Internal modules (prefixed with `_`) and implementation details > not listed in a module's `__all__` are considered private and may change diff --git a/docs/api/lsp.md b/docs/api/lsp.md new file mode 100644 index 000000000..a19be750e --- /dev/null +++ b/docs/api/lsp.md @@ -0,0 +1,302 @@ +# `cleveragents.lsp` — Language Server Protocol Integration + +The `lsp` package integrates Language Server Protocol (LSP) servers into +CleverAgents actors, exposing code intelligence capabilities (diagnostics, +completions, hover, definitions, and more) as callable tools. + +See [ADR-027](../adr/ADR-027-language-server-protocol.md) for the design +rationale and [ADR-040](../adr/ADR-040-lsp-resource-types.md) for LSP +resource types. + +--- + +## Overview + +```python +from cleveragents.lsp import ( + LspRegistry, + LspRuntime, + LspServerConfig, + LspCapability, + LspTransport, + LspToolAdapter, + LspLifecycleManager, + LspClient, + LspBinding, + LanguageDiscovery, +) + +# Register a language server +registry = LspRegistry() +registry.register(LspServerConfig( + name="local/pyright", + description="Pyright static type checker for Python", + languages=["python"], + command="pyright-langserver", + args=["--stdio"], + capabilities=[LspCapability.DIAGNOSTICS, LspCapability.HOVER, + LspCapability.COMPLETIONS, LspCapability.DEFINITIONS], +)) + +# Start the runtime and query diagnostics +runtime = LspRuntime(registry=registry) +runtime.start_server("local/pyright", workspace_path="/path/to/project") +diagnostics = runtime.get_diagnostics("local/pyright", file_path="src/main.py") +``` + +--- + +## `LspTransport` + +```python +class LspTransport(StrEnum): + STDIO = "stdio" + TCP = "tcp" +``` + +Transport mechanism for LSP server communication. + +| Value | Description | +|-------|-------------| +| `STDIO` | Default. Server spawned as subprocess; communicates via stdin/stdout JSON-RPC. | +| `TCP` | Server listens on a TCP port; runtime connects as a client. | + +--- + +## `LspCapability` + +```python +class LspCapability(StrEnum): + DIAGNOSTICS = "diagnostics" + HOVER = "hover" + COMPLETIONS = "completions" + DEFINITIONS = "definitions" + REFERENCES = "references" + RENAME = "rename" + CODE_ACTIONS = "code_actions" + FORMATTING = "formatting" + SIGNATURE_HELP = "signature_help" + DOCUMENT_SYMBOLS = "document_symbols" + WORKSPACE_SYMBOLS = "workspace_symbols" +``` + +LSP capabilities that can be exposed as callable tools to actors. Each +member maps to a Language Server Protocol feature that `LspToolAdapter` +can present through the standard tool interface. + +--- + +## `LspServerConfig` + +Pydantic model — registration record for a language server. + +| Field | Type | Description | +|-------|------|-------------| +| `name` | `str` | Namespaced server name, e.g. `local/pyright`. | +| `description` | `str` | Human-readable description. | +| `languages` | `list[str]` | Programming languages served, e.g. `["python"]`. | +| `command` | `str` | Executable to start the server. | +| `args` | `list[str]` | Additional CLI arguments. | +| `transport` | `LspTransport` | Communication transport (default: `STDIO`). | +| `env` | `dict[str, str]` | Extra environment variables for the server process. | +| `capabilities` | `list[LspCapability]` | LSP capabilities this server exposes. | +| `initialization` | `dict[str, Any]` | Custom `initializationOptions` for the LSP handshake. | +| `workspace_settings` | `dict[str, Any]` | Settings sent via `workspace/didChangeConfiguration`. | + +--- + +## `LspBinding` + +Pydantic model — binds an LSP server to an actor graph node. + +| Field | Type | Description | +|-------|------|-------------| +| `server` | `str` | Namespaced server name to bind. | +| `capabilities` | `list[LspCapability]` | Subset of capabilities to expose to this node. | + +Used in actor YAML under `nodes..lsp_sources`: + +```yaml +nodes: + analyzer: + model: gpt-4o + lsp_sources: + - server: local/pyright + capabilities: [diagnostics, hover, completions] +``` + +--- + +## `LspRegistry` + +Thread-safe, in-memory registry for `LspServerConfig` objects. + +```python +registry = LspRegistry() +``` + +### Methods + +| Method | Signature | Description | +|--------|-----------|-------------| +| `register` | `(config: LspServerConfig) → None` | Register a server config. Raises `ValueError` if already registered. | +| `get` | `(name: str) → LspServerConfig \| None` | Look up a config by namespaced name. | +| `get_required` | `(name: str) → LspServerConfig` | Like `get` but raises `LspServerNotFoundError` if missing. | +| `list_all` | `() → list[LspServerConfig]` | Return all registered configs. | +| `unregister` | `(name: str) → None` | Remove a config. Raises `LspServerNotFoundError` if not found. | + +--- + +## `LspLifecycleManager` + +Thread-safe manager for running LSP server processes with reference counting. + +```python +manager = LspLifecycleManager() +``` + +Multiple actors can share a single server process. The process is only +stopped when the last reference is released. + +### Methods + +| Method | Signature | Description | +|--------|-----------|-------------| +| `start_server` | `(config, workspace_path) → LspClient` | Start or acquire a reference to a server. | +| `stop_server` | `(name: str) → None` | Release a reference; shuts down when count reaches zero. | +| `get_client` | `(name: str) → LspClient` | Get the client for a running server. | +| `restart_server` | `(name: str) → LspClient` | Stop and restart a server (e.g. after crash). | +| `list_running` | `() → list[str]` | Return names of all currently running servers. | +| `is_running` | `(name: str) → bool` | Check whether a server is running and healthy. | + +**Deadlock-safe restart:** `restart_server` releases the internal lock +before performing blocking I/O (stop + start), preventing deadlocks when +called from within a lock-holding context. + +--- + +## `LspRuntime` + +High-level runtime that combines `LspRegistry` and `LspLifecycleManager` +for managing language servers end-to-end. + +```python +runtime = LspRuntime(registry=registry) +``` + +### Constructor + +| Parameter | Type | Default | Description | +|-----------|------|---------|-------------| +| `registry` | `LspRegistry \| None` | `None` | Registry to use (creates one if not provided). | +| `lifecycle_manager` | `LspLifecycleManager \| None` | `None` | Lifecycle manager (creates one if not provided). | + +### Methods + +| Method | Signature | Description | +|--------|-----------|-------------| +| `start_server` | `(name, workspace_path) → None` | Start a server by name. | +| `stop_server` | `(name) → None` | Release a server reference. | +| `get_diagnostics` | `(name, file_path, content?) → list[dict]` | Retrieve diagnostics for a file. | +| `get_hover` | `(name, file_path, line, character) → dict \| None` | Get hover info at a position. | +| `get_completions` | `(name, file_path, line, character) → list[dict]` | Get completions at a position. | +| `get_definitions` | `(name, file_path, line, character) → list[dict]` | Go to definition. | +| `get_references` | `(name, file_path, line, character) → list[dict]` | Find all references. | +| `is_server_running` | `(name) → bool` | Check if a server is running and healthy. | + +--- + +## `LspToolAdapter` + +Bridges LSP capabilities to the CleverAgents tool interface so actors can +discover and invoke language-server features as standard tools. + +```python +adapter = LspToolAdapter(runtime=runtime) +tool_specs = adapter.get_tool_specs(config) +``` + +### Constructor + +| Parameter | Type | Description | +|-----------|------|-------------| +| `runtime` | `LspRuntime \| None` | Runtime for real LSP calls. Without one, handlers raise `LspNotAvailableError`. | + +### Methods + +| Method | Signature | Description | +|--------|-----------|-------------| +| `get_tool_specs` | `(config: LspServerConfig) → list[ToolSpec]` | Generate tool specs for all capabilities in *config*. | +| `get_handler` | `(config, capability) → Callable` | Get the handler function for a specific capability. | + +**Generated tool names** follow the pattern `lsp:/`, +e.g. `lsp:local/pyright/diagnostics`. + +--- + +## `LspClient` + +Low-level JSON-RPC 2.0 client for the LSP protocol. Normally you interact +with `LspRuntime` rather than `LspClient` directly. + +```python +client = LspClient(transport, server_name="local/pyright") +client.initialize(workspace_path="/path/to/project") +result = client.send_request("textDocument/hover", params={...}) +client.shutdown() +``` + +--- + +## `LanguageDiscovery` + +Discovers installed language servers on the system. + +```python +discovery = LanguageDiscovery() +servers = discovery.discover() # returns list[LspServerConfig] +``` + +Scans `$PATH` for known language server executables and returns +pre-configured `LspServerConfig` objects for each one found. + +--- + +## Error Types + +| Exception | Description | +|-----------|-------------| +| `LspError` | Base class for all LSP errors. | +| `LspNotAvailableError` | Raised when no `LspRuntime` is configured. | +| `LspServerNotFoundError` | Raised when a server name is not registered or not running. | + +--- + +## Usage in Actor YAML + +```yaml +name: local/code-analyst +entry_node: analyzer +nodes: + analyzer: + model: gpt-4o + tool_sources: [builtin] + lsp_sources: + - server: local/pyright + capabilities: [diagnostics, hover, completions, definitions] + - server: local/typescript-language-server + capabilities: [diagnostics, completions] +``` + +The actor compiler resolves `lsp_sources` at startup, registers the +servers, and wires the generated tool specs into the node's tool set. + +--- + +## Related Documentation + +- [ADR-027 Language Server Protocol Integration](../adr/ADR-027-language-server-protocol.md) +- [ADR-040 LSP Resource Types](../adr/ADR-040-lsp-resource-types.md) +- [LSP Reference](../reference/lsp.md) +- [LSP Lifecycle Restart](../reference/lsp_lifecycle_restart.md) +- [LSP Stub](../reference/lsp_stub.md) diff --git a/mkdocs.yml b/mkdocs.yml index 2bb05974c..fede4725d 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -23,6 +23,8 @@ nav: - Configuration: api/config.md - AI Providers: api/providers.md - TUI: api/tui.md + - LSP Integration: api/lsp.md + - ACMS / UKO Vocabularies: api/acms.md - Modules: - Shell Safety: modules/shell-safety.md - UKO Provenance Tracking: modules/uko-provenance.md