forked from HAL9000/cleveragents-core
b51df2ee0f
## Summary This PR adds Kubernetes Helm deployment support for the CleverAgents server. Closes #928 ### What this PR includes - Helm chart under `k8s/` with Deployment, Service, optional Ingress, ConfigMap, ServiceAccount, Secrets, NOTES, and optional Redis subchart configuration. - Multi-stage `Dockerfile.server` for server runtime deployment. - Deployment-focused docs in `k8s/README.md`. - Behave + Robot + benchmark coverage for chart/deployment wiring. ### Review fixes applied (cycle 11 — hurui200320 review #2687) **Critical fix:** 1. **CI SHA256 checksum verification fixed** — All 3 Helm install blocks (`unit_tests`, `integration_tests`, `helm` jobs) now save the tarball using its original filename (`helm-v3.16.4-linux-amd64.tar.gz`) instead of `helm.tgz`, so the `.sha256sum` file can correctly locate and verify it. This was causing all 3 CI Helm jobs to fail with "No such file or directory". **Major fixes (test coverage gaps):** 2. **405 Allow header now tested** — The existing "POST to known path returns method not allowed" scenario now asserts that the `Allow: GET` header is present. Added a second 405 scenario testing `POST /live` for broader `_KNOWN_PATHS` coverage (review item #11). 3. **Security-hardening headers now tested** — New scenario "HTTP responses include security-hardening headers" verifies `content-length`, `x-content-type-options: nosniff`, and `cache-control: no-store` headers are present on HTTP responses. 4. **Lifespan warning logging now tested** — New scenario "Unrecognised lifespan message type logs warning and continues" queues `lifespan.startup` → `lifespan.bogus` → `lifespan.shutdown` and verifies: (a) the app completes the lifespan cycle cleanly, (b) a warning is logged mentioning the unrecognised type. ### Review fixes applied (cycle 10) **Critical/Major fixes (from hurui200320's prior REQUEST_CHANGES):** 1. **Rebased branch onto master** — Removed merge commit per CONTRIBUTING.md rebase-only policy. Clean linear history restored. 2. **Fixed commit message body** — Replaced literal `\n` sequences with actual newlines. `ISSUES CLOSED: #928` footer is now on its own line after a blank separator. 3. **Moved uvicorn import to module top level** — `from uvicorn import run as uvicorn_run` is now at the top of `src/cleveragents/cli/commands/server.py` per Import Guidelines. Updated test mock target from `uvicorn.run` to `cleveragents.cli.commands.server.uvicorn_run`. 4. **Added SHA256 checksum verification** — All three Helm CLI install blocks in `.forgejo/workflows/ci.yml` now download and verify `helm.sha256sum` before extracting the binary. **Minor fixes:** 5. **Added `Dockerfile.server` build to CI** — New "Build Docker image (Server)" step in the `docker` job validates the server Dockerfile. 6. **ASGI 405 Method Not Allowed** — Known paths (`/`, `/live`, `/ready`, `/health`) now return 405 with `Allow: GET` header for non-GET methods, per RFC 9110 §15.5.6. Added `_KNOWN_PATHS` frozenset. 7. **WebSocket close protocol fix** — App now calls `await receive()` to consume the `websocket.connect` event before closing. Changed close code from 1000 (Normal Closure) to 1008 (Policy Violation). 8. **Lifespan handler logging** — Unrecognised lifespan message types are now logged as warnings instead of silently consumed. 9. **Security-hardening headers** — `_send_response` now includes `content-length`, `x-content-type-options: nosniff`, and `cache-control: no-store` on all HTTP responses. 10. **`.dockerignore` credential patterns** — Added `*.pem`, `*.key`, `*.p12`, `*.pfx`, `credentials*.json`. 11. **`--log-level` validation** — Constrained to `click.Choice(["critical", "error", "warning", "info", "debug", "trace"])` for clean CLI validation errors. 12. **Reverted unrelated semgrep pre-commit change** — `pass_filenames` and `entry` restored to original values per atomic commit hygiene. 13. **Removed unused `ReceiveCallable` type alias** and `Callable`/`Awaitable` imports from `asgi_app_steps.py`. 14. **Fixed redundant `shutil.which("helm")` check** — `_skip_if_helm_missing` now returns `bool` to eliminate the duplicate check in `_render_chart`. 15. **Improved test deque error handling** — Lifespan test receive mock now raises descriptive `AssertionError` instead of opaque `IndexError`. 16. **Scope type dispatch** — Changed `if/if/if` to `if/elif/elif` for mutually exclusive ASGI scope types. 17. **Dockerfile.server base image** — Standardised to `python:3.13-slim` (floating minor) consistent with CLI Dockerfile. 18. **Dockerfile layer caching** — Split `uv pip install build` and `python -m build` into separate `RUN` instructions. 19. **Removed extraneous double blank line** in Dockerfile.server. ### Deferred items (acknowledged, not in scope) - PodDisruptionBudget, HorizontalPodAutoscaler, NetworkPolicy — Follow-up for production hardening. - `appVersion: "1.0.0"` placeholder — Needs tracking issue for release versioning alignment. - Readiness probe with downstream dependency checks — Documented limitation. - Cross-system test for probe paths matching ASGI routes — Test enhancement. - Improved benchmarks (helm template timing vs PyYAML parsing) — Benchmark quality improvement. - CI DRY violation (Helm install 3×) — Code quality improvement, consider composite action. - File length limits exceeded (`k8s_helm_chart_steps.py` 551 lines, `helper_k8s_helm_chart.py` 678 lines) — Non-blocking, can be split in follow-up. - `runAsGroup: 1000` in pod security context — Defense-in-depth improvement. - HEAD method support on known paths — RFC compliance, does not affect K8s probes. - `click.Choice` log-level validation via CLI runner test — Test gap. ### Scope note: status-check CI gate The `status-check` job now includes `integration_tests`, `e2e_tests`, and `helm` in its `needs` list. The `helm` job is new in this PR. The `integration_tests` and `e2e_tests` additions fix previously-missing gate checks — included here since this PR modifies both of those jobs to install Helm. ### Quality gates - `nox -e lint` ✅ - `nox -e typecheck` ✅ - `nox -e unit_tests` ✅ (12,321 scenarios passed, 4 skipped) - `nox -e integration_tests` — 3 pre-existing failures in unrelated areas (plan correction, resource types) - `nox -e e2e_tests` — pre-existing failures (LLM API keys not available in local env) - `nox -e coverage_report` ✅ (**97.7%**) Reviewed-on: cleveragents/cleveragents-core#1085 Reviewed-by: Jeffrey Phillips Freeman <jeffrey.freeman@cleverthis.com> Co-authored-by: Brent E. Edwards <brent.edwards@cleverthis.com> Co-committed-by: Brent E. Edwards <brent.edwards@cleverthis.com>
436 lines
14 KiB
Python
436 lines
14 KiB
Python
"""Step definitions for server CLI coverage boost scenarios.
|
|
|
|
Targets uncovered lines in src/cleveragents/cli/commands/server.py:
|
|
- Lines 58-59: resolve_server_mode exception branches
|
|
- Lines 132-142: server_connect rich format Panel rendering
|
|
- Lines 157-204: server_status command (entire function)
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import contextlib
|
|
import re
|
|
from io import StringIO
|
|
from typing import Any
|
|
from unittest.mock import MagicMock, patch
|
|
|
|
import typer
|
|
from behave import given, then, when
|
|
from behave.runner import Context
|
|
from rich.console import Console
|
|
from typer.testing import CliRunner
|
|
|
|
from cleveragents.application.services.config_service import (
|
|
ConfigLevel,
|
|
ResolvedValue,
|
|
)
|
|
|
|
_ANSI_ESCAPE_RE = re.compile(r"\x1B\[[0-?]*[ -/]*[@-~]")
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Helpers
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
def _make_resolved(key: str, value: Any) -> ResolvedValue:
|
|
"""Build a ResolvedValue with DEFAULT source."""
|
|
return ResolvedValue(key=key, value=value, source=ConfigLevel.DEFAULT)
|
|
|
|
|
|
def _build_mock_svc(
|
|
resolve_map: dict[str, Any] | None = None,
|
|
raise_map: dict[str, type[Exception]] | None = None,
|
|
) -> MagicMock:
|
|
"""Build a mock ConfigService whose resolve() returns controlled values.
|
|
|
|
Parameters
|
|
----------
|
|
resolve_map:
|
|
``{key: value}`` — the mock returns a ``ResolvedValue`` with this value.
|
|
raise_map:
|
|
``{key: ExceptionClass}`` — the mock raises the given exception.
|
|
"""
|
|
resolve_map = resolve_map or {}
|
|
raise_map = raise_map or {}
|
|
|
|
svc = MagicMock()
|
|
|
|
def _resolve(key: str, **_kw: Any) -> ResolvedValue:
|
|
if key in raise_map:
|
|
raise raise_map[key](f"mock error for {key}")
|
|
val = resolve_map.get(key)
|
|
return _make_resolved(key, val)
|
|
|
|
svc.resolve.side_effect = _resolve
|
|
svc.read_config.return_value = {}
|
|
svc.write_config.return_value = None
|
|
return svc
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Given — mock config services
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@given("a mock config service that raises ValueError on resolve")
|
|
def step_mock_svc_valueerror(context: Context) -> None:
|
|
context.mock_svc = _build_mock_svc(raise_map={"server.url": ValueError})
|
|
|
|
|
|
@given("a mock config service that raises KeyError on resolve")
|
|
def step_mock_svc_keyerror(context: Context) -> None:
|
|
context.mock_svc = _build_mock_svc(raise_map={"server.url": KeyError})
|
|
|
|
|
|
@given("a mock config service that accepts writes")
|
|
def step_mock_svc_writes(context: Context) -> None:
|
|
context.mock_svc = _build_mock_svc()
|
|
|
|
|
|
@given("a mock config service that resolves with no server config")
|
|
def step_mock_svc_no_config(context: Context) -> None:
|
|
context.mock_svc = _build_mock_svc(
|
|
resolve_map={
|
|
"server.url": None,
|
|
"server.namespace": None,
|
|
"server.tls-verify": None,
|
|
},
|
|
)
|
|
|
|
|
|
@given("a mock config service that resolves with full server config")
|
|
def step_mock_svc_full_config(context: Context) -> None:
|
|
context.mock_svc = _build_mock_svc(
|
|
resolve_map={
|
|
"server.url": "https://configured.example.com",
|
|
"server.namespace": "production",
|
|
"server.tls-verify": False,
|
|
},
|
|
)
|
|
|
|
|
|
@given("a mock config service where namespace resolve raises ValueError")
|
|
def step_mock_svc_ns_error(context: Context) -> None:
|
|
context.mock_svc = _build_mock_svc(
|
|
resolve_map={
|
|
"server.url": "https://ns-error.example.com",
|
|
"server.tls-verify": True,
|
|
},
|
|
raise_map={"server.namespace": ValueError},
|
|
)
|
|
|
|
|
|
@given("a mock config service where tls-verify resolve raises KeyError")
|
|
def step_mock_svc_tls_error(context: Context) -> None:
|
|
context.mock_svc = _build_mock_svc(
|
|
resolve_map={
|
|
"server.url": "https://tls-error.example.com",
|
|
"server.namespace": "default",
|
|
},
|
|
raise_map={"server.tls-verify": KeyError},
|
|
)
|
|
|
|
|
|
@given("a mock config service where all resolves raise exceptions")
|
|
def step_mock_svc_all_errors(context: Context) -> None:
|
|
context.mock_svc = _build_mock_svc(
|
|
raise_map={
|
|
"server.url": ValueError,
|
|
"server.namespace": KeyError,
|
|
"server.tls-verify": ValueError,
|
|
},
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# When — resolve_server_mode
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@when("I call resolve_server_mode with the mock")
|
|
def step_call_resolve_mode(context: Context) -> None:
|
|
from cleveragents.cli.commands.server import resolve_server_mode
|
|
|
|
with patch(
|
|
"cleveragents.cli.commands.server._get_config_service",
|
|
return_value=context.mock_svc,
|
|
):
|
|
context.server_mode_result = resolve_server_mode()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# When — server_connect with rich format
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@when('I call server_connect with url "{url}" and rich format')
|
|
def step_call_connect_rich(context: Context, url: str) -> None:
|
|
from cleveragents.cli.commands.server import server_connect
|
|
|
|
buf = StringIO()
|
|
capture_console = Console(file=buf, width=200, no_color=True)
|
|
with (
|
|
patch(
|
|
"cleveragents.cli.commands.server._get_config_service",
|
|
return_value=context.mock_svc,
|
|
),
|
|
patch(
|
|
"cleveragents.cli.commands.server.console",
|
|
capture_console,
|
|
),
|
|
):
|
|
server_connect(server_url=url, namespace="default", tls_verify=True, fmt="rich")
|
|
|
|
context.captured_console_output = buf.getvalue()
|
|
|
|
|
|
@when(
|
|
'I call server_connect with url "{url}" namespace "{ns}" tls_verify false and rich format'
|
|
)
|
|
def step_call_connect_rich_ns_tls(context: Context, url: str, ns: str) -> None:
|
|
from cleveragents.cli.commands.server import server_connect
|
|
|
|
buf = StringIO()
|
|
capture_console = Console(file=buf, width=200, no_color=True)
|
|
with (
|
|
patch(
|
|
"cleveragents.cli.commands.server._get_config_service",
|
|
return_value=context.mock_svc,
|
|
),
|
|
patch(
|
|
"cleveragents.cli.commands.server.console",
|
|
capture_console,
|
|
),
|
|
):
|
|
server_connect(server_url=url, namespace=ns, tls_verify=False, fmt="rich")
|
|
|
|
context.captured_console_output = buf.getvalue()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# When — server_status
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@when('I call server_status with format "{fmt}"')
|
|
def step_call_status(context: Context, fmt: str) -> None:
|
|
from cleveragents.cli.commands.server import server_status
|
|
|
|
echo_buf = StringIO()
|
|
console_buf = StringIO()
|
|
capture_console = Console(file=console_buf, width=200, no_color=True)
|
|
|
|
def _capture_echo(msg: str = "", **_kw: Any) -> None:
|
|
echo_buf.write(str(msg))
|
|
echo_buf.write("\n")
|
|
|
|
# We need to also patch resolve_server_mode since it calls _get_config_service
|
|
# internally. We compute the expected mode from the mock_svc.
|
|
mock_svc = context.mock_svc
|
|
|
|
# Determine what resolve_server_mode would return given this mock
|
|
try:
|
|
rv = mock_svc.resolve("server.url")
|
|
if rv.value is not None and str(rv.value).strip():
|
|
expected_mode = "stubbed"
|
|
else:
|
|
expected_mode = "disabled"
|
|
except (ValueError, KeyError):
|
|
expected_mode = "disabled"
|
|
|
|
stdout_buf = StringIO()
|
|
with (
|
|
patch(
|
|
"cleveragents.cli.commands.server._get_config_service",
|
|
return_value=mock_svc,
|
|
),
|
|
patch(
|
|
"cleveragents.cli.commands.server.resolve_server_mode",
|
|
return_value=expected_mode,
|
|
),
|
|
patch(
|
|
"cleveragents.cli.commands.server.typer.echo",
|
|
side_effect=_capture_echo,
|
|
),
|
|
patch(
|
|
"cleveragents.cli.commands.server.console",
|
|
capture_console,
|
|
),
|
|
contextlib.redirect_stdout(stdout_buf),
|
|
):
|
|
server_status(fmt=fmt)
|
|
|
|
echoed = echo_buf.getvalue().strip()
|
|
context.echoed_output = echo_buf.getvalue() if echoed else stdout_buf.getvalue()
|
|
context.captured_console_output = console_buf.getvalue()
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# Then — assertions
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@then('the server mode should be "{expected}"')
|
|
def step_assert_mode(context: Context, expected: str) -> None:
|
|
assert context.server_mode_result == expected, (
|
|
f"Expected mode '{expected}', got '{context.server_mode_result}'"
|
|
)
|
|
|
|
|
|
@then('the captured console output should contain "{text}"')
|
|
def step_assert_console_contains(context: Context, text: str) -> None:
|
|
output = getattr(context, "captured_console_output", "")
|
|
assert text in output, f"Expected '{text}' in console output:\n{output!r}"
|
|
|
|
|
|
@then('the echoed output should contain "{text}"')
|
|
def step_assert_echo_contains(context: Context, text: str) -> None:
|
|
output = getattr(context, "echoed_output", "")
|
|
assert text.lower() in output.lower(), (
|
|
f"Expected '{text}' in echoed output:\n{output!r}"
|
|
)
|
|
|
|
|
|
# ---------------------------------------------------------------------------
|
|
# server_serve — uvicorn.run mapping assertions
|
|
# ---------------------------------------------------------------------------
|
|
|
|
|
|
@given("uvicorn run is patched for server serve assertions")
|
|
def step_patch_uvicorn_run(context: Context) -> None:
|
|
uvicorn_run_patcher = patch(
|
|
"cleveragents.cli.commands.server.uvicorn_run",
|
|
)
|
|
context.uvicorn_run_patcher = uvicorn_run_patcher
|
|
context.uvicorn_run_mock = uvicorn_run_patcher.start()
|
|
context.add_cleanup(uvicorn_run_patcher.stop)
|
|
|
|
|
|
@given("uvicorn run is unavailable for server serve assertions")
|
|
def step_patch_uvicorn_unavailable(context: Context) -> None:
|
|
uvicorn_run_patcher = patch(
|
|
"cleveragents.cli.commands.server.uvicorn_run",
|
|
None,
|
|
)
|
|
context.uvicorn_run_patcher = uvicorn_run_patcher
|
|
uvicorn_run_patcher.start()
|
|
context.add_cleanup(uvicorn_run_patcher.stop)
|
|
|
|
|
|
@when(
|
|
'I call server_serve with app "{app_target}" host "{host}" port {port:d} workers {workers:d} and log level "{log_level}"'
|
|
)
|
|
def step_call_server_serve_explicit(
|
|
context: Context,
|
|
app_target: str,
|
|
host: str,
|
|
port: int,
|
|
workers: int,
|
|
log_level: str,
|
|
) -> None:
|
|
from cleveragents.cli.commands import server as server_commands
|
|
|
|
server_commands.server_serve(
|
|
app_target=app_target,
|
|
host=host,
|
|
port=port,
|
|
workers=workers,
|
|
log_level=log_level,
|
|
)
|
|
|
|
|
|
@when("I call server_serve with default arguments")
|
|
def step_call_server_serve_defaults(context: Context) -> None:
|
|
from cleveragents.cli.commands import server as server_commands
|
|
|
|
server_commands.server_serve()
|
|
|
|
|
|
@when("I call server_serve with default arguments and capture failure")
|
|
def step_call_server_serve_defaults_capture_failure(context: Context) -> None:
|
|
from cleveragents.cli.commands import server as server_commands
|
|
|
|
buf = StringIO()
|
|
capture_console = Console(file=buf, width=200, no_color=True)
|
|
|
|
context.server_serve_exit = None
|
|
with patch("cleveragents.cli.commands.server.console", capture_console):
|
|
try:
|
|
server_commands.server_serve()
|
|
except typer.Exit as exc:
|
|
context.server_serve_exit = exc
|
|
|
|
context.captured_console_output = buf.getvalue()
|
|
|
|
|
|
@when('I invoke server_serve through the CLI with log level "{log_level}"')
|
|
def step_invoke_server_serve_cli(context: Context, log_level: str) -> None:
|
|
from cleveragents.cli.commands import server as server_commands
|
|
|
|
runner = CliRunner()
|
|
context.cli_result = runner.invoke(
|
|
server_commands.app,
|
|
["serve", "--log-level", log_level],
|
|
)
|
|
|
|
|
|
@when('I invoke server_serve through the CLI with invalid log level "{log_level}"')
|
|
def step_invoke_server_serve_cli_invalid(context: Context, log_level: str) -> None:
|
|
from cleveragents.cli.commands import server as server_commands
|
|
|
|
runner = CliRunner()
|
|
context.cli_result = runner.invoke(
|
|
server_commands.app,
|
|
["serve", "--log-level", log_level],
|
|
)
|
|
|
|
|
|
@then(
|
|
'uvicorn run should be called with app "{app_target}" host "{host}" port {port:d} workers {workers:d} and log level "{log_level}"'
|
|
)
|
|
def step_assert_server_serve_mapping(
|
|
context: Context,
|
|
app_target: str,
|
|
host: str,
|
|
port: int,
|
|
workers: int,
|
|
log_level: str,
|
|
) -> None:
|
|
run_mock = context.uvicorn_run_mock
|
|
run_mock.assert_called_once_with(
|
|
app_target,
|
|
host=host,
|
|
port=port,
|
|
workers=workers,
|
|
log_level=log_level,
|
|
)
|
|
|
|
|
|
@then("server_serve should exit with code {code:d}")
|
|
def step_assert_server_serve_exit_code(context: Context, code: int) -> None:
|
|
exit_exc = getattr(context, "server_serve_exit", None)
|
|
assert isinstance(exit_exc, typer.Exit), (
|
|
f"Expected typer.Exit, got {type(exit_exc).__name__}: {exit_exc!r}"
|
|
)
|
|
exit_code = exit_exc.exit_code
|
|
assert exit_code is not None, "Expected typer.Exit to carry an exit code"
|
|
assert exit_code == code, f"Expected exit code {code}, got {exit_code}"
|
|
|
|
|
|
@then("the CLI invocation should fail with exit code {code:d}")
|
|
def step_assert_cli_exit_code(context: Context, code: int) -> None:
|
|
result = context.cli_result
|
|
assert result.exit_code == code, (
|
|
f"Expected CLI exit code {code}, got {result.exit_code}. "
|
|
f"Output:\n{result.output}"
|
|
)
|
|
|
|
|
|
@then('the server CLI invocation output should contain "{text}"')
|
|
def step_assert_cli_output_contains(context: Context, text: str) -> None:
|
|
result = context.cli_result
|
|
normalized_output = _ANSI_ESCAPE_RE.sub("", result.output)
|
|
assert text in normalized_output, (
|
|
f"Expected '{text}' in CLI output, got:\n{result.output!r}"
|
|
)
|