Files
temp/features/steps/server_cli_coverage_boost_steps.py
T
brent.edwards b51df2ee0f feat(server): add Kubernetes Helm chart for server deployment (#1085)
## Summary

This PR adds Kubernetes Helm deployment support for the CleverAgents server.

Closes #928

### What this PR includes

- Helm chart under `k8s/` with Deployment, Service, optional Ingress, ConfigMap,
  ServiceAccount, Secrets, NOTES, and optional Redis subchart configuration.
- Multi-stage `Dockerfile.server` for server runtime deployment.
- Deployment-focused docs in `k8s/README.md`.
- Behave + Robot + benchmark coverage for chart/deployment wiring.

### Review fixes applied (cycle 11 — hurui200320 review #2687)

**Critical fix:**

1. **CI SHA256 checksum verification fixed** — All 3 Helm install blocks (`unit_tests`, `integration_tests`, `helm` jobs) now save the tarball using its original filename (`helm-v3.16.4-linux-amd64.tar.gz`) instead of `helm.tgz`, so the `.sha256sum` file can correctly locate and verify it. This was causing all 3 CI Helm jobs to fail with "No such file or directory".

**Major fixes (test coverage gaps):**

2. **405 Allow header now tested** — The existing "POST to known path returns method not allowed" scenario now asserts that the `Allow: GET` header is present. Added a second 405 scenario testing `POST /live` for broader `_KNOWN_PATHS` coverage (review item #11).
3. **Security-hardening headers now tested** — New scenario "HTTP responses include security-hardening headers" verifies `content-length`, `x-content-type-options: nosniff`, and `cache-control: no-store` headers are present on HTTP responses.
4. **Lifespan warning logging now tested** — New scenario "Unrecognised lifespan message type logs warning and continues" queues `lifespan.startup` → `lifespan.bogus` → `lifespan.shutdown` and verifies: (a) the app completes the lifespan cycle cleanly, (b) a warning is logged mentioning the unrecognised type.

### Review fixes applied (cycle 10)

**Critical/Major fixes (from hurui200320's prior REQUEST_CHANGES):**

1. **Rebased branch onto master** — Removed merge commit per CONTRIBUTING.md rebase-only policy. Clean linear history restored.
2. **Fixed commit message body** — Replaced literal `\n` sequences with actual newlines. `ISSUES CLOSED: #928` footer is now on its own line after a blank separator.
3. **Moved uvicorn import to module top level** — `from uvicorn import run as uvicorn_run` is now at the top of `src/cleveragents/cli/commands/server.py` per Import Guidelines. Updated test mock target from `uvicorn.run` to `cleveragents.cli.commands.server.uvicorn_run`.
4. **Added SHA256 checksum verification** — All three Helm CLI install blocks in `.forgejo/workflows/ci.yml` now download and verify `helm.sha256sum` before extracting the binary.

**Minor fixes:**

5. **Added `Dockerfile.server` build to CI** — New "Build Docker image (Server)" step in the `docker` job validates the server Dockerfile.
6. **ASGI 405 Method Not Allowed** — Known paths (`/`, `/live`, `/ready`, `/health`) now return 405 with `Allow: GET` header for non-GET methods, per RFC 9110 §15.5.6. Added `_KNOWN_PATHS` frozenset.
7. **WebSocket close protocol fix** — App now calls `await receive()` to consume the `websocket.connect` event before closing. Changed close code from 1000 (Normal Closure) to 1008 (Policy Violation).
8. **Lifespan handler logging** — Unrecognised lifespan message types are now logged as warnings instead of silently consumed.
9. **Security-hardening headers** — `_send_response` now includes `content-length`, `x-content-type-options: nosniff`, and `cache-control: no-store` on all HTTP responses.
10. **`.dockerignore` credential patterns** — Added `*.pem`, `*.key`, `*.p12`, `*.pfx`, `credentials*.json`.
11. **`--log-level` validation** — Constrained to `click.Choice(["critical", "error", "warning", "info", "debug", "trace"])` for clean CLI validation errors.
12. **Reverted unrelated semgrep pre-commit change** — `pass_filenames` and `entry` restored to original values per atomic commit hygiene.
13. **Removed unused `ReceiveCallable` type alias** and `Callable`/`Awaitable` imports from `asgi_app_steps.py`.
14. **Fixed redundant `shutil.which("helm")` check** — `_skip_if_helm_missing` now returns `bool` to eliminate the duplicate check in `_render_chart`.
15. **Improved test deque error handling** — Lifespan test receive mock now raises descriptive `AssertionError` instead of opaque `IndexError`.
16. **Scope type dispatch** — Changed `if/if/if` to `if/elif/elif` for mutually exclusive ASGI scope types.
17. **Dockerfile.server base image** — Standardised to `python:3.13-slim` (floating minor) consistent with CLI Dockerfile.
18. **Dockerfile layer caching** — Split `uv pip install build` and `python -m build` into separate `RUN` instructions.
19. **Removed extraneous double blank line** in Dockerfile.server.

### Deferred items (acknowledged, not in scope)

- PodDisruptionBudget, HorizontalPodAutoscaler, NetworkPolicy — Follow-up for production hardening.
- `appVersion: "1.0.0"` placeholder — Needs tracking issue for release versioning alignment.
- Readiness probe with downstream dependency checks — Documented limitation.
- Cross-system test for probe paths matching ASGI routes — Test enhancement.
- Improved benchmarks (helm template timing vs PyYAML parsing) — Benchmark quality improvement.
- CI DRY violation (Helm install 3×) — Code quality improvement, consider composite action.
- File length limits exceeded (`k8s_helm_chart_steps.py` 551 lines, `helper_k8s_helm_chart.py` 678 lines) — Non-blocking, can be split in follow-up.
- `runAsGroup: 1000` in pod security context — Defense-in-depth improvement.
- HEAD method support on known paths — RFC compliance, does not affect K8s probes.
- `click.Choice` log-level validation via CLI runner test — Test gap.

### Scope note: status-check CI gate

The `status-check` job now includes `integration_tests`, `e2e_tests`, and `helm` in its `needs` list. The `helm` job is new in this PR. The `integration_tests` and `e2e_tests` additions fix previously-missing gate checks — included here since this PR modifies both of those jobs to install Helm.

### Quality gates

- `nox -e lint` 
- `nox -e typecheck` 
- `nox -e unit_tests`  (12,321 scenarios passed, 4 skipped)
- `nox -e integration_tests` — 3 pre-existing failures in unrelated areas (plan correction, resource types)
- `nox -e e2e_tests` — pre-existing failures (LLM API keys not available in local env)
- `nox -e coverage_report`  (**97.7%**)

Reviewed-on: cleveragents/cleveragents-core#1085
Reviewed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com>
Co-authored-by: Brent E. Edwards <brent.edwards@cleverthis.com>
Co-committed-by: Brent E. Edwards <brent.edwards@cleverthis.com>
2026-03-27 23:53:22 +00:00

436 lines
14 KiB
Python

"""Step definitions for server CLI coverage boost scenarios.
Targets uncovered lines in src/cleveragents/cli/commands/server.py:
- Lines 58-59: resolve_server_mode exception branches
- Lines 132-142: server_connect rich format Panel rendering
- Lines 157-204: server_status command (entire function)
"""
from __future__ import annotations
import contextlib
import re
from io import StringIO
from typing import Any
from unittest.mock import MagicMock, patch
import typer
from behave import given, then, when
from behave.runner import Context
from rich.console import Console
from typer.testing import CliRunner
from cleveragents.application.services.config_service import (
ConfigLevel,
ResolvedValue,
)
_ANSI_ESCAPE_RE = re.compile(r"\x1B\[[0-?]*[ -/]*[@-~]")
# ---------------------------------------------------------------------------
# Helpers
# ---------------------------------------------------------------------------
def _make_resolved(key: str, value: Any) -> ResolvedValue:
"""Build a ResolvedValue with DEFAULT source."""
return ResolvedValue(key=key, value=value, source=ConfigLevel.DEFAULT)
def _build_mock_svc(
resolve_map: dict[str, Any] | None = None,
raise_map: dict[str, type[Exception]] | None = None,
) -> MagicMock:
"""Build a mock ConfigService whose resolve() returns controlled values.
Parameters
----------
resolve_map:
``{key: value}`` — the mock returns a ``ResolvedValue`` with this value.
raise_map:
``{key: ExceptionClass}`` — the mock raises the given exception.
"""
resolve_map = resolve_map or {}
raise_map = raise_map or {}
svc = MagicMock()
def _resolve(key: str, **_kw: Any) -> ResolvedValue:
if key in raise_map:
raise raise_map[key](f"mock error for {key}")
val = resolve_map.get(key)
return _make_resolved(key, val)
svc.resolve.side_effect = _resolve
svc.read_config.return_value = {}
svc.write_config.return_value = None
return svc
# ---------------------------------------------------------------------------
# Given — mock config services
# ---------------------------------------------------------------------------
@given("a mock config service that raises ValueError on resolve")
def step_mock_svc_valueerror(context: Context) -> None:
context.mock_svc = _build_mock_svc(raise_map={"server.url": ValueError})
@given("a mock config service that raises KeyError on resolve")
def step_mock_svc_keyerror(context: Context) -> None:
context.mock_svc = _build_mock_svc(raise_map={"server.url": KeyError})
@given("a mock config service that accepts writes")
def step_mock_svc_writes(context: Context) -> None:
context.mock_svc = _build_mock_svc()
@given("a mock config service that resolves with no server config")
def step_mock_svc_no_config(context: Context) -> None:
context.mock_svc = _build_mock_svc(
resolve_map={
"server.url": None,
"server.namespace": None,
"server.tls-verify": None,
},
)
@given("a mock config service that resolves with full server config")
def step_mock_svc_full_config(context: Context) -> None:
context.mock_svc = _build_mock_svc(
resolve_map={
"server.url": "https://configured.example.com",
"server.namespace": "production",
"server.tls-verify": False,
},
)
@given("a mock config service where namespace resolve raises ValueError")
def step_mock_svc_ns_error(context: Context) -> None:
context.mock_svc = _build_mock_svc(
resolve_map={
"server.url": "https://ns-error.example.com",
"server.tls-verify": True,
},
raise_map={"server.namespace": ValueError},
)
@given("a mock config service where tls-verify resolve raises KeyError")
def step_mock_svc_tls_error(context: Context) -> None:
context.mock_svc = _build_mock_svc(
resolve_map={
"server.url": "https://tls-error.example.com",
"server.namespace": "default",
},
raise_map={"server.tls-verify": KeyError},
)
@given("a mock config service where all resolves raise exceptions")
def step_mock_svc_all_errors(context: Context) -> None:
context.mock_svc = _build_mock_svc(
raise_map={
"server.url": ValueError,
"server.namespace": KeyError,
"server.tls-verify": ValueError,
},
)
# ---------------------------------------------------------------------------
# When — resolve_server_mode
# ---------------------------------------------------------------------------
@when("I call resolve_server_mode with the mock")
def step_call_resolve_mode(context: Context) -> None:
from cleveragents.cli.commands.server import resolve_server_mode
with patch(
"cleveragents.cli.commands.server._get_config_service",
return_value=context.mock_svc,
):
context.server_mode_result = resolve_server_mode()
# ---------------------------------------------------------------------------
# When — server_connect with rich format
# ---------------------------------------------------------------------------
@when('I call server_connect with url "{url}" and rich format')
def step_call_connect_rich(context: Context, url: str) -> None:
from cleveragents.cli.commands.server import server_connect
buf = StringIO()
capture_console = Console(file=buf, width=200, no_color=True)
with (
patch(
"cleveragents.cli.commands.server._get_config_service",
return_value=context.mock_svc,
),
patch(
"cleveragents.cli.commands.server.console",
capture_console,
),
):
server_connect(server_url=url, namespace="default", tls_verify=True, fmt="rich")
context.captured_console_output = buf.getvalue()
@when(
'I call server_connect with url "{url}" namespace "{ns}" tls_verify false and rich format'
)
def step_call_connect_rich_ns_tls(context: Context, url: str, ns: str) -> None:
from cleveragents.cli.commands.server import server_connect
buf = StringIO()
capture_console = Console(file=buf, width=200, no_color=True)
with (
patch(
"cleveragents.cli.commands.server._get_config_service",
return_value=context.mock_svc,
),
patch(
"cleveragents.cli.commands.server.console",
capture_console,
),
):
server_connect(server_url=url, namespace=ns, tls_verify=False, fmt="rich")
context.captured_console_output = buf.getvalue()
# ---------------------------------------------------------------------------
# When — server_status
# ---------------------------------------------------------------------------
@when('I call server_status with format "{fmt}"')
def step_call_status(context: Context, fmt: str) -> None:
from cleveragents.cli.commands.server import server_status
echo_buf = StringIO()
console_buf = StringIO()
capture_console = Console(file=console_buf, width=200, no_color=True)
def _capture_echo(msg: str = "", **_kw: Any) -> None:
echo_buf.write(str(msg))
echo_buf.write("\n")
# We need to also patch resolve_server_mode since it calls _get_config_service
# internally. We compute the expected mode from the mock_svc.
mock_svc = context.mock_svc
# Determine what resolve_server_mode would return given this mock
try:
rv = mock_svc.resolve("server.url")
if rv.value is not None and str(rv.value).strip():
expected_mode = "stubbed"
else:
expected_mode = "disabled"
except (ValueError, KeyError):
expected_mode = "disabled"
stdout_buf = StringIO()
with (
patch(
"cleveragents.cli.commands.server._get_config_service",
return_value=mock_svc,
),
patch(
"cleveragents.cli.commands.server.resolve_server_mode",
return_value=expected_mode,
),
patch(
"cleveragents.cli.commands.server.typer.echo",
side_effect=_capture_echo,
),
patch(
"cleveragents.cli.commands.server.console",
capture_console,
),
contextlib.redirect_stdout(stdout_buf),
):
server_status(fmt=fmt)
echoed = echo_buf.getvalue().strip()
context.echoed_output = echo_buf.getvalue() if echoed else stdout_buf.getvalue()
context.captured_console_output = console_buf.getvalue()
# ---------------------------------------------------------------------------
# Then — assertions
# ---------------------------------------------------------------------------
@then('the server mode should be "{expected}"')
def step_assert_mode(context: Context, expected: str) -> None:
assert context.server_mode_result == expected, (
f"Expected mode '{expected}', got '{context.server_mode_result}'"
)
@then('the captured console output should contain "{text}"')
def step_assert_console_contains(context: Context, text: str) -> None:
output = getattr(context, "captured_console_output", "")
assert text in output, f"Expected '{text}' in console output:\n{output!r}"
@then('the echoed output should contain "{text}"')
def step_assert_echo_contains(context: Context, text: str) -> None:
output = getattr(context, "echoed_output", "")
assert text.lower() in output.lower(), (
f"Expected '{text}' in echoed output:\n{output!r}"
)
# ---------------------------------------------------------------------------
# server_serve — uvicorn.run mapping assertions
# ---------------------------------------------------------------------------
@given("uvicorn run is patched for server serve assertions")
def step_patch_uvicorn_run(context: Context) -> None:
uvicorn_run_patcher = patch(
"cleveragents.cli.commands.server.uvicorn_run",
)
context.uvicorn_run_patcher = uvicorn_run_patcher
context.uvicorn_run_mock = uvicorn_run_patcher.start()
context.add_cleanup(uvicorn_run_patcher.stop)
@given("uvicorn run is unavailable for server serve assertions")
def step_patch_uvicorn_unavailable(context: Context) -> None:
uvicorn_run_patcher = patch(
"cleveragents.cli.commands.server.uvicorn_run",
None,
)
context.uvicorn_run_patcher = uvicorn_run_patcher
uvicorn_run_patcher.start()
context.add_cleanup(uvicorn_run_patcher.stop)
@when(
'I call server_serve with app "{app_target}" host "{host}" port {port:d} workers {workers:d} and log level "{log_level}"'
)
def step_call_server_serve_explicit(
context: Context,
app_target: str,
host: str,
port: int,
workers: int,
log_level: str,
) -> None:
from cleveragents.cli.commands import server as server_commands
server_commands.server_serve(
app_target=app_target,
host=host,
port=port,
workers=workers,
log_level=log_level,
)
@when("I call server_serve with default arguments")
def step_call_server_serve_defaults(context: Context) -> None:
from cleveragents.cli.commands import server as server_commands
server_commands.server_serve()
@when("I call server_serve with default arguments and capture failure")
def step_call_server_serve_defaults_capture_failure(context: Context) -> None:
from cleveragents.cli.commands import server as server_commands
buf = StringIO()
capture_console = Console(file=buf, width=200, no_color=True)
context.server_serve_exit = None
with patch("cleveragents.cli.commands.server.console", capture_console):
try:
server_commands.server_serve()
except typer.Exit as exc:
context.server_serve_exit = exc
context.captured_console_output = buf.getvalue()
@when('I invoke server_serve through the CLI with log level "{log_level}"')
def step_invoke_server_serve_cli(context: Context, log_level: str) -> None:
from cleveragents.cli.commands import server as server_commands
runner = CliRunner()
context.cli_result = runner.invoke(
server_commands.app,
["serve", "--log-level", log_level],
)
@when('I invoke server_serve through the CLI with invalid log level "{log_level}"')
def step_invoke_server_serve_cli_invalid(context: Context, log_level: str) -> None:
from cleveragents.cli.commands import server as server_commands
runner = CliRunner()
context.cli_result = runner.invoke(
server_commands.app,
["serve", "--log-level", log_level],
)
@then(
'uvicorn run should be called with app "{app_target}" host "{host}" port {port:d} workers {workers:d} and log level "{log_level}"'
)
def step_assert_server_serve_mapping(
context: Context,
app_target: str,
host: str,
port: int,
workers: int,
log_level: str,
) -> None:
run_mock = context.uvicorn_run_mock
run_mock.assert_called_once_with(
app_target,
host=host,
port=port,
workers=workers,
log_level=log_level,
)
@then("server_serve should exit with code {code:d}")
def step_assert_server_serve_exit_code(context: Context, code: int) -> None:
exit_exc = getattr(context, "server_serve_exit", None)
assert isinstance(exit_exc, typer.Exit), (
f"Expected typer.Exit, got {type(exit_exc).__name__}: {exit_exc!r}"
)
exit_code = exit_exc.exit_code
assert exit_code is not None, "Expected typer.Exit to carry an exit code"
assert exit_code == code, f"Expected exit code {code}, got {exit_code}"
@then("the CLI invocation should fail with exit code {code:d}")
def step_assert_cli_exit_code(context: Context, code: int) -> None:
result = context.cli_result
assert result.exit_code == code, (
f"Expected CLI exit code {code}, got {result.exit_code}. "
f"Output:\n{result.output}"
)
@then('the server CLI invocation output should contain "{text}"')
def step_assert_cli_output_contains(context: Context, text: str) -> None:
result = context.cli_result
normalized_output = _ANSI_ESCAPE_RE.sub("", result.output)
assert text in normalized_output, (
f"Expected '{text}' in CLI output, got:\n{result.output!r}"
)