cleveragents-core/docs/api/lsp.md

# `cleveragents.lsp` — Language Server Protocol Integration

The `cleveragents.lsp` package wires Language Server Protocol (LSP) features
into CleverAgents. It provides:

- Pydantic models describing servers, transports, and bindings.
- A thread-safe registry for language server configurations.
- Runtime orchestration (start, health-check, restart, stop) with automatic
  reference counting.
- A synchronous protocol client and lifecycle manager around stdio transports.
- A tool adapter that exposes LSP capabilities to the generic tool system.
- A multi-layer language discovery engine that selects servers for files.

All exports listed below are available directly from `cleveragents.lsp`.

---

## Quick start

```python
from cleveragents.lsp import (
    LanguageDiscovery,
    LspCapability,
    LspRegistry,
    LspRuntime,
    LspServerConfig,
    LspToolAdapter,
)

registry = LspRegistry()
registry.register(
    LspServerConfig(
        name="local/pyright",
        description="Pyright language server",
        command="pyright-langserver",
        languages=["python"],
        capabilities=[
            LspCapability.DIAGNOSTICS,
            LspCapability.COMPLETIONS,
            LspCapability.HOVER,
        ],
    )
)

runtime = LspRuntime(registry=registry)
runtime.start_server("local/pyright", workspace_path="/workspace")

adapter = LspToolAdapter(runtime)
pyright_tools = adapter.generate_tool_specs(registry.get_or_raise("local/pyright"))
print([spec["name"] for spec in pyright_tools])
# ['local/pyright/diagnostics', 'local/pyright/completions', 'local/pyright/hover']

language = LanguageDiscovery().detect_file_language("/workspace/app/models.py")
print(language)  # 'python'
```

---

## Models

### `LspTransport`

`StrEnum` describing how the runtime communicates with a server:

- `STDIO` (default) — spawn the server and communicate via stdin/stdout JSON-RPC.
- `TCP` — connect to a server listening on a TCP socket.

### `LspCapability`

Enum of LSP features that can be surfaced as ToolSpecs. Members map directly to
protocol operations:

| Capability | Description |
|------------|-------------|
| `DIAGNOSTICS` | Fetch diagnostics, warnings, and errors for a file. |
| `HOVER` | Retrieve hover information (type hints, docs) at a position. |
| `COMPLETIONS` | Request completion candidates at a position. |
| `DEFINITIONS` | Go to definition for a symbol. |
| `REFERENCES` | Find references to a symbol. |
| `RENAME` | Request a rename operation across the workspace. |
| `CODE_ACTIONS` | Obtain code actions for a range. |
| `FORMATTING` | Format a document or selection. |
| `SIGNATURE_HELP` | Fetch call signature help. |
| `DOCUMENT_SYMBOLS` | List symbols defined in a document. |
| `WORKSPACE_SYMBOLS` | Search for symbols across the workspace. |

> **Note:** The runtime currently provides concrete handlers for diagnostics,
> completions, hover, and definitions. Other capabilities raise
> `LspNotAvailableError` until their protocol flows are implemented.

### `LspServerConfig`

Pydantic model describing a language server.

| Field | Type | Description |
|-------|------|-------------|
| `name` | `str` | Namespaced identifier (`namespace/name`); validated to contain a slash. |
| `description` | `str` | Optional human-readable description. |
| `languages` | `list[str]` | Lowercased language identifiers served by the process. |
| `command` | `str` | Executable used to start the server. |
| `args` | `list[str]` | Extra CLI arguments. |
| `transport` | `LspTransport` | Communication mode (`STDIO` by default). |
| `env` | `dict[str, str]` | Environment overrides for the process. |
| `capabilities` | `list[LspCapability]` | Enabled protocol features. |
| `initialization` | `dict[str, Any]` | `initialize` request `initializationOptions`. |
| `workspace_settings` | `dict[str, Any]` | Settings pushed after initialization. |

Helper properties:

- `namespace` — portion before the last `/`.
- `short_name` — portion after the last `/`.
- `to_dict()` — JSON-compatible serialisation respecting model validators.

### `LspBinding`

Binds a server to an actor graph node. Used by the compiler/runtime to know
which servers to activate. Validates that `lsp_server_name` is namespaced and
supports optional per-node language filters.

---

## Registry and discovery

### `LspRegistry`

Thread-safe registry for `LspServerConfig` objects. Public methods acquire a
re-entrant lock, making concurrent registration and lookup safe.

- `register(config)` — add a new server; rejects duplicates.
- `get(name)` — return a config or `None` when missing.
- `get_or_raise(name)` — raise `LspServerNotFoundError` if absent.
- `list_servers(namespace=None, language=None)` — filter by namespace prefix or
  supported language, sorted by name.
- `remove(name)` — unregister a server; returns `True` if removed.

### `LanguageDiscovery`

Four-layer language detection engine used by the runtime and compiler. Layers
are consulted in priority order:

1. File extension (`.py`, `.tsx`, …) via a built-in lookup table.
2. Shebang inspection for ambiguous or extension-less files.
3. Optional UKO (Universal Knowledge Ontology) classifications supplied at
   construction.
4. Explicit project-level language declarations passed to the constructor.

Key methods:

- `detect_file_language(path)` — returns a language ID (falls back to
  `"plaintext"`). Results are cached per path.
- `detect_directory_languages(path)` — walk a tree and return sorted unique
  languages (ignoring `plaintext`).
- `invalidate(path)` / `invalidate_all()` — manage the detection cache.
- `get_servers_for_language(language, configs)` — helper to match registered
  servers to a detected language.

---

## Runtime orchestration

### `LspLifecycleManager`

Manages running server processes with reference counting. Guarantees that the
underlying transport is started only once per server/workspace combination and
shutdown occurs when the last reference is released. Provides:

- `start_server(config, workspace_path)` — start or reuse a server, returning an
  `LspClient`.
- `stop_server(name)` — release a reference; raises `LspServerNotFoundError` if
  the server was never acquired.
- `health_check(name)` — return `True` if the process is alive.
- `restart_server(name)` — restart a crashed server while preserving reference
  counts.
- `stop_all()` / `list_running()` — maintenance helpers for graceful shutdown
  and observability.

### `LspClient`

Synchronous JSON-RPC client layered on top of `StdioTransport`. Exposes typed
methods for the protocol flows currently needed by CleverAgents:

- `initialize(workspace_path, initialization_options=None)` and `shutdown()` for
  lifecycle management.
- Document operations: `did_open`, `did_close`, `get_diagnostics`,
  `get_completions`, `get_hover`, `get_definitions`.
- Accessors `is_initialized` and `server_capabilities` for telemetry or testing.

Internally the client keeps a bounded queue of notifications, correlates
responses by ID, and raises `LspError` on timeouts or protocol failures.

### `LspRuntime`

High-level facade combining the registry and lifecycle manager. It performs
input validation, logs significant events, and exposes convenience methods:

- `start_server(name, workspace_path)` / `stop_server(name)` — manage process
  lifecycle.
- `get_diagnostics`, `get_completions`, `get_hover`, `get_definitions` — delegate
  to the underlying `LspClient`, reading file contents and translating between
  filesystem paths and LSP URIs.
- `activate_bindings(bindings, workspace_path)` and `deactivate_bindings(bindings)` —
  start/stop all servers referenced by `LspBinding` objects produced during actor
  compilation.
- `stop_all()` — forward to `LspLifecycleManager.stop_all()`.

If a server crashes, `LspRuntime` attempts to restart it automatically before
servicing the next request.

### `StdioTransport`

Process wrapper that launches language servers via subprocess, reads/writes
JSON-RPC messages, and exposes `is_alive`, `start()`, `stop()`, `send_message()`,
`read_message()`. It is the default transport used by the lifecycle manager and
client.

---

## Tool adapter

### `LspToolAdapter`

Generates tool specifications from `LspServerConfig.capabilities` so the
standard tool runtime can invoke language-server functionality.

```python
from cleveragents.lsp import LspToolAdapter

adapter = LspToolAdapter(runtime)
for spec in adapter.generate_tool_specs(config):
    tool_registry.register(spec)
```

- When constructed with an `LspRuntime`, capability handlers delegate to runtime
  methods (diagnostics, completions, hover, definitions today). Missing runtime
  capabilities raise `LspNotAvailableError` with a structured payload.
- Without a runtime (local/offline mode), generated handlers always raise
  `LspNotAvailableError`, signalling that remote execution is required.
- Each spec includes a minimal JSON Schema input contract tailored to the
  capability (file-only, position-based, rename, or query-based).

---

## Error types

The package exports structured exceptions for callers to handle:

- `LspError` — generic protocol or transport failure.
- `LspNotAvailableError` — capability invoked without an available runtime.
- `LspServerNotFoundError` — registry or lifecycle lookup of an unknown server.

Catch these errors in higher layers to provide user-facing remediation (e.g.
retry prompts, error banners in the TUI, or fallback tool behaviour).