diff --git a/docs/api/acms.md b/docs/api/acms.md new file mode 100644 index 000000000..1b873debc --- /dev/null +++ b/docs/api/acms.md @@ -0,0 +1,362 @@ +# `cleveragents.acms` — Advanced Context Management System + +The `acms` package provides UKO (Universal Knowledge Ontology) vocabulary +support for the Advanced Context Management System. It contains Layer 2 +paradigm vocabulary specializations and Layer 3 technology-specific vocabulary +extensions, along with the `DetailLevelMap` inheritance mechanism for resolving +named detail levels across the ontology hierarchy. + +For the full ACMS pipeline and context assembly, see +[`cleveragents.application.services`](../reference/acms.md). +For the UKO runtime, see [`docs/reference/uko_runtime.md`](../reference/uko_runtime.md). + +--- + +## Overview + +The `acms` package re-exports the public API of `cleveragents.acms.uko`: + +```python +from cleveragents.acms import ( + # Vocabulary registry + VocabularyRegistry, + + # Base UKO types + UKOClass, + UKOProperty, + UKOVocabulary, + Layer2Dependency, + ProvenanceInfo, + + # Vocabulary helpers + VocabularyClass, + VocabularyProperty, + ParadigmVocabulary, + + # Python vocabulary + PYTHON_VOCABULARY, + PYTHON_DETAIL_LEVELS, + PythonClass, + PythonFunction, + PythonModule, + PythonDecorator, + PythonTypeStub, + + # TypeScript vocabulary + TYPESCRIPT_VOCABULARY, + TYPESCRIPT_DETAIL_LEVELS, + TypeScriptClass, + TypeScriptFunction, + TypeScriptInterface, + TypeScriptModule, + + # Rust vocabulary + RUST_VOCABULARY, + RUST_DETAIL_LEVELS, + RustStruct, + RustTrait, + RustImpl, + RustFunction, + RustDeriveAttribute, + + # Java vocabulary + JAVA_VOCABULARY, + JAVA_DETAIL_LEVELS, + JavaClass, + JavaInterface, + JavaMethod, + JavaAnnotation, + JavaCheckedException, + + # Detail level maps + CODE_DETAIL_LEVEL_MAP, + FUNC_DETAIL_LEVEL_MAP, + OO_DETAIL_LEVEL_MAP, + PROC_DETAIL_LEVEL_MAP, + DetailLevelMapBuilder, + + # Utility functions + build_detail_level_map, + build_effective_map, + get_func_vocabulary, + get_oo_vocabulary, + get_proc_vocabulary, + resolve_detail_level, + + # Errors + DuplicateVocabularyError, +) +``` + +--- + +## Core Types + +### `ProvenanceInfo` + +Provenance contract attached to every vocabulary class and property. + +| Field | Type | Description | +|-------|------|-------------| +| `source_resource` | `str` | URI or ULID of the originating resource | +| `source_path` | `str` | File path within the source artifact | +| `source_range` | `str` | `"startLine:startCol-endLine:endCol"` range string | +| `valid_from` | `datetime` | UTC timestamp when the node became valid | +| `is_current` | `bool` | Indicates whether this node is the current revision | + +### `Layer2Dependency` + +Declares that a vocabulary node depends on a parent-layer URI. + +| Field | Type | Description | +|-------|------|-------------| +| `uri` | `str` | Parent-layer URI (e.g. `"https://…/uko/code#TypeDefinition"`) | +| `layer` | `int` | UKO layer number (1–3) | +| `prefix` | `str` | Parent-layer prefix such as `"uko-oo:"` | +| `description` | `str` | Human-readable description of the dependency | + +### `UKOClass` + +OWL class definition for a vocabulary. + +| Field | Type | Description | +|-------|------|-------------| +| `uri` | `str` | Full IRI (validated to use http/https) | +| `label` | `str` | Human-readable label | +| `comment` | `str` | Description | +| `subclass_of` | `tuple[str, ...]` | Parent class IRIs (inheritance chain) | +| `provenance` | `ProvenanceInfo | None` | Optional provenance metadata | + +### `UKOProperty` + +OWL property definition for a vocabulary. + +| Field | Type | Description | +|-------|------|-------------| +| `uri` | `str` | Full IRI | +| `label` | `str` | Human-readable label | +| `comment` | `str` | Description | +| `domain` | `tuple[str, ...]` | Domain class IRIs | +| `range_uri` | `str` | Range class or datatype IRI | +| `is_object_property` | `bool` | `True` for object properties, `False` for datatype properties | +| `provenance` | `ProvenanceInfo | None` | Optional provenance metadata | + +### `UKOVocabulary` + +Container for a vocabulary's classes, properties, and Layer 2 dependencies. + +| Field | Type | Description | +|-------|------|-------------| +| `namespace` | `str` | Full namespace IRI | +| `prefix` | `str` | Namespace prefix (e.g. `"uko-py:"`) | +| `layer` | `int` | UKO layer number (technology vocabularies use `3`) | +| `label` | `str` | Human-readable name | +| `comment` | `str` | Description | +| `parent_prefixes` | `tuple[str, ...]` | Parent Layer 2 prefixes extended | +| `classes` | `tuple[UKOClass, ...]` | OWL class definitions | +| `properties` | `tuple[UKOProperty, ...]` | OWL property definitions | +| `layer2_dependencies` | `tuple[Layer2Dependency, ...]` | Required Layer 2 URIs | +| `inserted_levels` | `tuple[tuple[str, str], ...]` | Named DetailLevelMap insertions | +| `provenance` | `ProvenanceInfo | None` | Optional provenance metadata | + +--- + +## Paradigm Vocabulary Helpers + +### `VocabularyClass` + +Layer 2 OWL class definition used by paradigm vocabularies. + +| Field | Type | Description | +|-------|------|-------------| +| `uri` | `str` | Full IRI for the class | +| `label` | `str` | `rdfs:label` value | +| `comment` | `str` | `rdfs:comment` text | +| `parent_uris` | `tuple[str, ...]` | `rdfs:subClassOf` parent IRIs | + +### `VocabularyProperty` + +Layer 2 OWL property definition. + +| Field | Type | Description | +|-------|------|-------------| +| `uri` | `str` | Full IRI for the property | +| `label` | `str` | `rdfs:label` value | +| `domain_uri` | `str` | `rdfs:domain` class IRI | +| `range_uri` | `str` | `rdfs:range` class IRI | +| `sub_property_of` | `str | None` | Optional `rdfs:subPropertyOf` parent | + +### `ParadigmVocabulary` + +Frozen Pydantic model describing a Layer 2 vocabulary. + +| Field | Type | Description | +|-------|------|-------------| +| `prefix` | `str` | Namespace prefix (must end with `:`) | +| `iri` | `str` | Full namespace IRI | +| `layer` | `int` | Ontology layer (always `2` for paradigm vocabularies) | +| `parent_domain` | `str` | Parent Layer 1 prefix (`"uko-code:"`) | +| `classes` | `tuple[VocabularyClass, ...]` | OWL class definitions | +| `properties` | `tuple[VocabularyProperty, ...]` | OWL property definitions | + +Helper methods `get_class(label)` and `get_property(label)` perform label-based +lookups. + +--- + +## Built-in Vocabularies + +Use `get_oo_vocabulary()`, `get_func_vocabulary()`, and `get_proc_vocabulary()` +to retrieve the Layer 2 paradigm vocabularies programmatically. Layer 3 +technology vocabularies are available via the constants listed below. + +### Python (`PYTHON_VOCABULARY`) + +Layer 3 vocabulary for Python source code analysis. + +| Class | IRI suffix | Description | +|-------|-----------|-------------| +| `PythonModule` | `Module` | A Python module (`.py` file) | +| `PythonClass` | `Class` | A Python class definition | +| `PythonFunction` | `Function` | A Python function or method | +| `PythonDecorator` | `Decorator` | A Python decorator | +| `PythonTypeStub` | `TypeStub` | A `.pyi` type stub | + +### TypeScript (`TYPESCRIPT_VOCABULARY`) + +Layer 3 vocabulary for TypeScript/JavaScript source code. + +| Class | Description | +|-------|-------------| +| `TypeScriptModule` | A TypeScript module | +| `TypeScriptClass` | A TypeScript class | +| `TypeScriptInterface` | A TypeScript interface | +| `TypeScriptFunction` | A TypeScript function | + +### Rust (`RUST_VOCABULARY`) + +Layer 3 vocabulary for Rust source code. + +| Class | Description | +|-------|-------------| +| `RustStruct` | A Rust struct | +| `RustTrait` | A Rust trait | +| `RustImpl` | A Rust impl block | +| `RustFunction` | A Rust function | +| `RustDeriveAttribute` | A `#[derive(...)]` attribute | + +### Java (`JAVA_VOCABULARY`) + +Layer 3 vocabulary for Java source code. + +| Class | Description | +|-------|-------------| +| `JavaClass` | A Java class | +| `JavaInterface` | A Java interface | +| `JavaMethod` | A Java method | +| `JavaAnnotation` | A Java annotation | +| `JavaCheckedException` | A checked exception class | + +--- + +## Detail Level Maps + +Detail level maps control how much information is included when rendering a +UKO node at a given depth. They follow a four-layer inheritance chain: +Layer 3 → Layer 2 → Layer 1 → Layer 0. + +### Built-in Maps + +| Constant | Description | +|----------|-------------| +| `OO_DETAIL_LEVEL_MAP` | Object-oriented paradigm levels (0–9) | +| `FUNC_DETAIL_LEVEL_MAP` | Functional paradigm levels | +| `PROC_DETAIL_LEVEL_MAP` | Procedural paradigm levels | +| `CODE_DETAIL_LEVEL_MAP` | Generic code levels | + +### `DetailLevelMapBuilder` + +Utility for constructing `DetailLevelMap` instances with insertions and +derivations. Most callers use the exported constants instead of instantiating +the builder directly. + +### `resolve_detail_level` + +Walk the inheritance chain to resolve a named detail level. + +```python +from cleveragents.acms import resolve_detail_level, PYTHON_DETAIL_LEVELS + +depth = resolve_detail_level("FULL_SOURCE", PYTHON_DETAIL_LEVELS) +# Returns the integer depth (0-9) for that level name +``` + +### `build_detail_level_map` + +Merge a parent map with child-level insertions. Both arguments are tuples of +`(name, depth)` pairs. + +```python +from cleveragents.acms import build_detail_level_map + +parent_levels = ( + ("MODULE_LISTING", 0), + ("FULL_SOURCE", 9), +) +insertions = (("MY_LEVEL", 5),) + +custom_map = build_detail_level_map(parent_levels, insertions) +``` + +### `build_effective_map` + +Helper that materialises a read-only `DetailLevelMap` structure with calculated +depths from a parent map and insertion tuples. Used internally by the built-in +vocabularies when combining insertions. + +--- + +## `VocabularyRegistry` + +Registry for all active UKO vocabularies. + +```python +from cleveragents.acms import VocabularyRegistry, PYTHON_VOCABULARY + +registry = VocabularyRegistry() +registry.register(PYTHON_VOCABULARY) +vocab = registry.get_by_prefix("uko-py:") +``` + +| Method | Description | +|--------|-------------| +| `register(vocab)` | Register a vocabulary; raises `DuplicateVocabularyError` on duplicate prefix or IRI | +| `unregister(prefix)` | Remove a vocabulary by prefix; returns `True` if removed | +| `get_by_prefix(prefix)` | Retrieve by short prefix (e.g. `"uko-oo:"`) | +| `get_by_iri(iri)` | Retrieve by full namespace IRI | +| `list_prefixes()` | Return registered prefixes in sorted order | +| `list_all()` | Return all registered vocabularies sorted by prefix | +| `__contains__(value)` | Check whether a prefix or IRI is registered | +| `__len__()` | Number of registered vocabularies | + +All entries returned by the registry are frozen `ParadigmVocabulary` instances; +callers should treat them as immutable snapshots. + +--- + +## Errors + +| Exception | Description | +|-----------|-------------| +| `DuplicateVocabularyError` | Raised when registering a vocabulary with an already-registered prefix | + +--- + +## Related Documentation + +- [UKO Runtime](../reference/uko_runtime.md) — query interface, inference engine, persistence +- [UKO Ontology](../reference/uko.md) — four-layer ontology specification +- [UKO Layer 2 Vocabularies](../reference/uko_layer2_paradigm_vocabularies.md) — paradigm vocabulary reference +- [UKO Layer 3 Vocabularies](../reference/uko_layer3_technology_vocabularies.md) — technology vocabulary reference +- [ACMS Pipeline](../reference/acms.md) — context assembly pipeline +- [Context Strategies](../reference/context_strategies.md) — built-in retrieval strategies diff --git a/docs/api/index.md b/docs/api/index.md index 4eee7341e..94c56eb22 100644 --- a/docs/api/index.md +++ b/docs/api/index.md @@ -14,6 +14,8 @@ classes, functions, and exceptions with signatures and usage examples. | [`cleveragents.skills`](skills.md) | Skill framework — schema, protocol, registry, discovery, and inline executor | | [`cleveragents.tool`](tool.md) | Tool runtime, lifecycle, registry, router, and container executor | | [`cleveragents.mcp`](mcp.md) | Model Context Protocol (MCP) adapter, client, registry, and sandbox | +| [`cleveragents.lsp`](lsp.md) | Language Server Protocol integration — registry, runtime, tool adapter, lifecycle | +| [`cleveragents.acms`](acms.md) | Advanced Context Management System — UKO vocabulary support, Layer 2/3 extensions | | [`cleveragents.resource`](resource.md) | Resource schema, handlers, and type-inheritance system | | [`cleveragents.config`](config.md) | Application settings, logging, metrics, and security scanning | | [`cleveragents.providers`](providers.md) | AI provider registry — discovery, selection, LLM factory, and capability metadata | diff --git a/docs/api/lsp.md b/docs/api/lsp.md new file mode 100644 index 000000000..5b7ed4242 --- /dev/null +++ b/docs/api/lsp.md @@ -0,0 +1,241 @@ +# `cleveragents.lsp` — Language Server Protocol Integration + +The `lsp` package provides the registry, runtime, tool adapter, server stub, +protocol client, transport, and lifecycle manager for integrating Language +Server Protocol (LSP) servers with CleverAgents actors. + +See [ADR-027](../adr/ADR-027-language-server-protocol.md) for design rationale +and [ADR-040](../adr/ADR-040-lsp-resource-types.md) for LSP resource types. + +--- + +## Quick Start + +```python +from cleveragents.lsp import ( + LspRegistry, + LspRuntime, + LspServerConfig, + LspCapability, + LspTransport, +) + +# Register a language server +registry = LspRegistry() +registry.register(LspServerConfig( + name="local/pyright", + description="Python type checker and language server", + languages=["python"], + command="pyright-langserver", + args=["--stdio"], + transport=LspTransport.STDIO, + capabilities=[LspCapability.DIAGNOSTICS, LspCapability.COMPLETIONS], +)) + +# Start and query +runtime = LspRuntime(registry=registry) +runtime.start_server("local/pyright", workspace_path="/path/to/project") +diagnostics = runtime.get_diagnostics("local/pyright", file_path="src/main.py") +``` + +--- + +## Enumerations + +### `LspTransport` + +Transport mechanism for LSP server communication. + +| Member | Value | Description | +|--------|-------|-------------| +| `STDIO` | `"stdio"` | Standard input/output (default). Server spawned as subprocess. | +| `TCP` | `"tcp"` | TCP socket connection. Server listens on a port. | + +### `LspCapability` + +LSP capabilities that can be exposed as callable tools to actors. + +| Member | Value | Description | +|--------|-------|-------------| +| `DIAGNOSTICS` | `"diagnostics"` | Syntax and type errors | +| `HOVER` | `"hover"` | Type information on hover | +| `COMPLETIONS` | `"completions"` | Code completion suggestions | +| `DEFINITIONS` | `"definitions"` | Go-to-definition | +| `REFERENCES` | `"references"` | Find all references | +| `RENAME` | `"rename"` | Symbol rename | +| `CODE_ACTIONS` | `"code_actions"` | Quick fixes and refactors | +| `FORMATTING` | `"formatting"` | Document formatting | +| `SIGNATURE_HELP` | `"signature_help"` | Function signature hints | +| `DOCUMENT_SYMBOLS` | `"document_symbols"` | Outline of symbols in a file | +| `WORKSPACE_SYMBOLS` | `"workspace_symbols"` | Workspace-wide symbol search | + +--- + +## Data Models + +### `LspServerConfig` + +Registration record for a language server. + +| Field | Type | Required | Description | +|-------|------|----------|-------------| +| `name` | `str` | Yes | Namespaced server name (e.g. `local/pyright`) | +| `description` | `str` | No | Human-readable description | +| `languages` | `list[str]` | No | Programming languages served | +| `command` | `str` | Yes | Executable command to start the server | +| `args` | `list[str]` | No | Additional CLI arguments | +| `transport` | `LspTransport` | No | Communication transport (default: `STDIO`) | +| `env` | `dict[str, str]` | No | Extra environment variables | +| `capabilities` | `list[LspCapability]` | No | Capabilities this server exposes | +| `initialization` | `dict[str, Any]` | No | Custom `initializationOptions` | +| `workspace_settings` | `dict[str, Any]` | No | Workspace settings sent after init | + +### `LspBinding` + +Binds an LSP server to an actor graph node. + +| Field | Type | Description | +|-------|------|-------------| +| `node_name` | `str` | Actor graph node name that owns the binding | +| `lsp_server_name` | `str` | Namespaced server name (must contain `/`) | +| `languages` | `list[str]` | Optional language allow-list (empty ⇒ all languages) | +| `auto_detect` | `bool` | Auto-detect language from file context when `True` | + +--- + +## `LspRegistry` + +Thread-safe, in-memory registry for LSP server configurations. + +```python +from cleveragents.lsp import LspRegistry, LspServerConfig + +registry = LspRegistry() +``` + +| Method | Signature | Description | +|--------|-----------|-------------| +| `register` | `(config: LspServerConfig) -> None` | Register a server config; raises `ValueError` if name already registered | +| `get` | `(name: str) -> LspServerConfig | None` | Retrieve by name; returns `None` if not found | +| `get_or_raise` | `(name: str) -> LspServerConfig` | Retrieve by name; raises `LspServerNotFoundError` if not found | +| `remove` | `(name: str) -> bool` | Remove a server config; returns `True` if removed | +| `list_servers` | `(namespace: str | None = None, language: str | None = None) -> list[LspServerConfig]` | List configs filtered by namespace and/or language | + +Use Python membership testing (`name in registry`) to check for registration; the +registry implements `__contains__` and `__len__` for introspection. + +--- + +## `LspRuntime` + +Manages the full lifecycle of language servers: start, query, health-check, +restart, and stop. + +```python +from cleveragents.lsp import LspRuntime, LspRegistry + +runtime = LspRuntime(registry=registry) +``` + +| Method | Signature | Description | +|--------|-----------|-------------| +| `start_server` | `(name: str, workspace_path: str) -> None` | Start a server; performs the LSP `initialize` handshake | +| `stop_server` | `(name: str) -> None` | Stop a running server (reference-counted) | +| `get_diagnostics` | `(name: str, file_path: str) -> list[dict[str, Any]]` | Retrieve diagnostics for a file | +| `get_completions` | `(name: str, file_path: str, line: int, column: int) -> list[Any]` | Code completions at a position | +| `get_hover` | `(name: str, file_path: str, line: int, column: int) -> dict[str, Any] | None` | Hover information at a position | +| `get_definitions` | `(name: str, file_path: str, line: int, column: int) -> list[dict[str, Any]]` | Go-to-definition locations | +| `activate_bindings` | `(bindings: list[Any], workspace_path: str) -> list[str]` | Start servers referenced by actor bindings | +| `deactivate_bindings` | `(bindings: list[Any]) -> None` | Release servers referenced by actor bindings | +| `stop_all` | `() -> None` | Stop all running servers | + +**Automatic restart:** If a server crashes during a query, the runtime +attempts one automatic restart before raising `LspError`. + +--- + +## `LspToolAdapter` + +Bridges LSP capabilities into the CleverAgents tool system. Each registered +capability becomes a callable tool in `ToolRegistry` with `source="lsp"`. + +```python +from cleveragents.lsp import LspToolAdapter, LspRuntime + +adapter = LspToolAdapter(runtime=runtime) +specs = adapter.generate_tool_specs(registry.get_or_raise("local/pyright")) +# specs: list[dict] for ToolRegistry.register_many(...) +``` + +When no runtime is supplied, tool handlers raise `LspNotAvailableError` +rather than silently returning empty results. + +--- + +## Low-level Exports + +The following utilities are re-exported for advanced scenarios: + +- `LanguageDiscovery` — Detects language IDs from file paths for runtime + operations. +- `LspServer` — Lightweight server wrapper used by the lifecycle manager. +- `StdioTransport` — Concrete transport implementation for stdio-based servers. + +Most applications interact with these indirectly via `LspRuntime` and +`LspLifecycleManager`. + +--- +## `LspLifecycleManager` + +Manages LSP server process spawning, health checks, and graceful shutdown. +Used internally by `LspRuntime`; rarely needed directly. + +```python +from cleveragents.lsp import LspLifecycleManager + +manager = LspLifecycleManager() +manager.start("local/pyright", config, workspace_path="/path/to/project") +manager.stop("local/pyright") +``` + +--- + +## `LspClient` + +Low-level JSON-RPC 2.0 client for LSP protocol communication. Used +internally by `LspRuntime`; prefer `LspRuntime` for most use cases. + +--- + +## Errors + +| Exception | Description | +|-----------|-------------| +| `LspError` | Base class for all LSP errors | +| `LspNotAvailableError` | Raised when LSP runtime is not configured | +| `LspServerNotFoundError` | Raised when a server name is not in the registry | + +--- + +## Actor YAML Integration + +Per-node LSP bindings in actor YAML control which server each node uses: + +```yaml +name: local/my-actor +nodes: + analyzer: + model: anthropic/claude-4-sonnet + lsp_bindings: + - server: local/pyright + capabilities: [diagnostics, completions, definitions] +``` + +--- + +## Related Documentation + +- [LSP Reference](../reference/lsp.md) — detailed protocol and lifecycle reference +- [LSP Resource Types](../reference/lsp_stub.md) — `lsp.*` resource type definitions +- [LSP Lifecycle Restart](../reference/lsp_lifecycle_restart.md) — automatic restart protocol +- [ADR-027](../adr/ADR-027-language-server-protocol.md) — design rationale diff --git a/docs/reference/diagnostics_checks.md b/docs/reference/diagnostics_checks.md index 16e762614..a70d65fbe 100644 --- a/docs/reference/diagnostics_checks.md +++ b/docs/reference/diagnostics_checks.md @@ -44,16 +44,20 @@ first-run) is accessible and writable. ### Provider API keys Checks whether each supported LLM provider has its API key configured. -One sub-check is emitted per provider. +One sub-check is emitted per provider. As of v3.8.0 all nine supported +providers are checked (extended from six in v3.7.0 — PR #3469). | Provider | Environment variable | Status when missing | |----------|---------------------|---------------------| | OpenAI | `OPENAI_API_KEY` | warn | | Anthropic | `ANTHROPIC_API_KEY` | warn | | Google | `GOOGLE_API_KEY` | warn | -| Google Gemini | `GEMINI_API_KEY` | warn | -| Hugging Face | `HF_TOKEN` | warn | +| Azure OpenAI | `AZURE_OPENAI_API_KEY` | warn | | OpenRouter | `OPENROUTER_API_KEY` | warn | +| Google Gemini | `GEMINI_API_KEY` | warn | +| Cohere | `COHERE_API_KEY` | warn | +| Groq | `GROQ_API_KEY` | warn | +| Together | `TOGETHER_API_KEY` | warn | | Status | Condition | Remediation | |--------|-----------|-------------| @@ -63,6 +67,9 @@ One sub-check is emitted per provider. Missing provider keys are warnings, not errors, because the system can operate with any single provider configured. +> **Note:** Hugging Face support (formerly `HF_TOKEN`) was removed in PR #3469; +> the diagnostics check list reflects the current provider implementations. + ### Disk space Checks available disk space on the volume containing the current working diff --git a/mkdocs.yml b/mkdocs.yml index 76940184b..1d093dd1c 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -19,6 +19,8 @@ nav: - Skills: api/skills.md - Tools: api/tool.md - MCP Adapter: api/mcp.md + - LSP Integration: api/lsp.md + - ACMS Vocabularies: api/acms.md - Resources: api/resource.md - Configuration: api/config.md - AI Providers: api/providers.md