test(session): add robot session model smoke tests
CI / lint (pull_request) Successful in 14s
CI / typecheck (pull_request) Successful in 27s
CI / security (pull_request) Successful in 25s
CI / quality (pull_request) Successful in 16s
CI / integration_tests (pull_request) Successful in 5m12s
CI / build (pull_request) Successful in 15s
CI / unit_tests (pull_request) Successful in 33m49s
CI / coverage (pull_request) Successful in 8m30s
CI / docker (pull_request) Successful in 1m16s

This commit is contained in:
2026-02-17 20:33:53 +00:00
parent 767d9d0f82
commit 4f32daa740
6 changed files with 420 additions and 18 deletions
+88
View File
@@ -0,0 +1,88 @@
# Session Domain Model
The **Session** domain model represents a persistent conversation thread
tied to an orchestrator actor. It maintains message history across plans
and serves as the user's natural-language interface.
## Key Classes
| Class | Description |
|-------|-------------|
| `Session` | Core session model with messages, token tracking, and export |
| `SessionMessage` | A single message within a session (ULID-identified, sequenced) |
| `MessageRole` | Enum: `user`, `assistant`, `system`, `tool` |
| `SessionTokenUsage` | Accumulated input/output token counts and cost |
## Source Location
```
src/cleveragents/domain/models/core/session.py
```
## Session Lifecycle
1. **Create** -- `Session(session_id=str(ULID()))` with optional `actor_name`
2. **Append messages** -- `session.append_message(role, content)` auto-sequences
3. **Query** -- `session.get_messages(limit, offset)` for paginated access
4. **Export** -- `session.as_export_dict()` produces a checksummed JSON dict
5. **CLI rendering** -- `session.as_cli_dict()` returns an `OrderedDict` for display
## Export Serialization Order
The `as_export_dict()` method produces keys in this fixed order:
```
schema_version, session_id, actor_name, namespace, messages,
linked_plan_ids, automation_level, token_usage, metadata,
created_at, updated_at, checksum
```
## Running Tests
### Behave (BDD unit tests)
```bash
nox -s unit_tests -- features/session_model.feature
```
### Robot Framework (smoke / integration tests)
```bash
nox -s integration_tests -- robot/session_model.robot
```
Or run the Robot suite directly:
```bash
robot --outputdir build/reports/robot robot/session_model.robot
```
### Robot Smoke Suite
The Robot smoke suite (`robot/session_model.robot`) validates:
- **Session creation** -- default fields (`session_id`, `namespace`, `is_empty`)
- **Message append ordering** -- auto-sequencing produces `[0, 1, 2, ...]`
- **Export dict completeness** -- all 12 required keys are present
- **CLI dict completeness** -- required display keys are present
- **Export key order** -- keys follow the canonical serialization order
The smoke tests delegate to `robot/helper_session_model.py`, which can also
be invoked standalone:
```bash
python robot/helper_session_model.py create
python robot/helper_session_model.py append-messages
python robot/helper_session_model.py export
python robot/helper_session_model.py cli-dict
python robot/helper_session_model.py export-key-order
```
### ASV Benchmarks
```bash
nox -s benchmark
```
Benchmark suites in `benchmarks/session_model_bench.py` cover validation
throughput, append performance, and serialization speed.
+12
View File
@@ -232,3 +232,15 @@ Feature: Session Domain Model
When I link plan "01ARZ3NDEKTSV4RRFFQ69G5FAV" to the session
And I export the session
Then the export dict should have key "linked_plan_ids"
# ---- Robot Smoke Alignment: Serialization Order ----
Scenario: Export dict keys are in expected serialization order
Given a session with some messages
When I export the session
Then the export dict keys should be in expected serialization order
Scenario: Export dict messages are ordered by sequence
Given a session with some messages
When I export the session
Then the export dict messages should be ordered by sequence
@@ -0,0 +1,41 @@
"""Step definitions for Robot smoke alignment scenarios in session model tests.
These steps mirror the expectations enforced by the Robot Framework smoke
suite in ``robot/session_model.robot`` so that Behave and Robot stay in sync.
"""
from behave import then
from behave.runner import Context
@then("the export dict keys should be in expected serialization order")
def session_model_check_export_key_order(context: Context) -> None:
"""Verify export dict keys follow the canonical serialization order."""
expected_order = [
"schema_version",
"session_id",
"actor_name",
"namespace",
"messages",
"linked_plan_ids",
"automation_level",
"token_usage",
"metadata",
"created_at",
"updated_at",
"checksum",
]
actual_keys = list(context.session_export_dict.keys())
assert actual_keys == expected_order, (
f"Expected export key order {expected_order}, got {actual_keys}"
)
@then("the export dict messages should be ordered by sequence")
def session_model_check_export_messages_sequence_order(context: Context) -> None:
"""Verify messages in the export dict are ordered by sequence."""
messages = context.session_export_dict["messages"]
sequences = [m["sequence"] for m in messages]
assert sequences == sorted(sequences), (
f"Export messages not ordered by sequence: {sequences}"
)
+18 -18
View File
@@ -2042,24 +2042,24 @@ No standalone Q0-Advanced commits planned. Advanced QA enhancements are bundled
- [X] Git [Jeff]: `git branch -d feature/m3-session-domain`
- [X] Quality [Jeff]: Verify coverage >=97% via `nox -s coverage_report`. Coverage 97% overall (session.py 98%).
- [ ] **COMMIT (Owner: Brent | Group: A7.domain.tests | Branch: feature/m3-session-domain-robot | Planned: Day 13 | Expected: Day 16) - Commit message: "test(session): add robot session model smoke tests"**
- [ ] Meta [Brent]: Only mark this commit complete after every subtask is done and `git commit -m "test(session): add robot session model smoke tests"` has executed.
- [ ] Git [Brent]: `git checkout master`
- [ ] Git [Brent]: `git pull origin master`
- [ ] Git [Brent]: `git checkout -b feature/m3-session-domain-robot`
- [ ] Git [Brent]: `git fetch origin && git merge origin/master` (run before final tests and before commit)
- [ ] Code [Brent]: Add `robot/session_model.robot` smoke tests for Session creation, message ordering, and serialization output.
- [ ] Docs [Brent]: Update `docs/reference/session_model.md` to mention the Robot smoke suite and how to run it.
- [ ] Tests (Behave) [Brent]: Add a scenario in `features/session_model.feature` that mirrors the Robot smoke expectations for serialization order.
- [ ] Tests (Robot) [Brent]: Add Robot suite that validates session model creation and export paths.
- [ ] Tests (ASV) [Brent]: Confirm `asv/benchmarks/session_model_bench.py` still passes after the new Robot suite is added.
- [ ] Quality [Brent]: Run `nox` (all default sessions, including benchmark), fix any errors if needed ensuring nox passes.
- [ ] Git [Brent]: `git add .`
- [ ] Git [Brent]: `git commit -m "test(session): add robot session model smoke tests"`
- [ ] Forgejo PR [Brent]: Open PR from `feature/m3-session-domain-robot` to `master` with description "Add Robot smoke tests for session domain model with docs and Behave alignment.".
- [ ] Git [Brent]: `git checkout master`
- [ ] Git [Brent]: `git branch -d feature/m3-session-domain-robot`
- [ ] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. If coverage is <97% then review the current unit test coverage report at `build/coverage.xml` and use it to write new Behave based unit tests to improve code coverage. Specifically, write Behave style unit tests that are descriptively named and specifically improves coverage on whichever file has the most uncovered lines by writing tests that will target the uncovered lines in the report. Once that is done rerun `nox -s coverage_report` to verify all tests pass and coverage is above >=97%. Only mark this as complete once coverage is >=97%, if not repeat this task as many times as is needed until coverage reaches >=97%.
- [X] **COMMIT (Owner: Brent | Group: A7.domain.tests | Branch: feature/m3-session-domain-robot | Planned: Day 13 | Expected: Day 16) - Commit message: "test(session): add robot session model smoke tests"**
- [X] Meta [Brent]: Only mark this commit complete after every subtask is done and `git commit -m "test(session): add robot session model smoke tests"` has executed.
- [X] Git [Brent]: `git checkout master`
- [X] Git [Brent]: `git pull origin master`
- [X] Git [Brent]: `git checkout -b feature/m3-session-domain-robot`
- [X] Git [Brent]: `git fetch origin && git merge origin/master` (run before final tests and before commit)
- [X] Code [Brent]: Add `robot/session_model.robot` smoke tests for Session creation, message ordering, and serialization output.
- [X] Docs [Brent]: Update `docs/reference/session_model.md` to mention the Robot smoke suite and how to run it.
- [X] Tests (Behave) [Brent]: Add a scenario in `features/session_model.feature` that mirrors the Robot smoke expectations for serialization order.
- [X] Tests (Robot) [Brent]: Add Robot suite that validates session model creation and export paths.
- [X] Tests (ASV) [Brent]: Confirm `asv/benchmarks/session_model_bench.py` still passes after the new Robot suite is added.
- [X] Quality [Brent]: Run `nox` (all default sessions, including benchmark), fix any errors if needed ensuring nox passes.
- [X] Git [Brent]: `git add .`
- [X] Git [Brent]: `git commit -m "test(session): add robot session model smoke tests"`
- [X] Forgejo PR [Brent]: Open PR from `feature/m3-session-domain-robot` to `master` with description "Add Robot smoke tests for session domain model with docs and Behave alignment.".
- [X] Git [Brent]: `git checkout master`
- [X] Git [Brent]: `git branch -d feature/m3-session-domain-robot`
- [X] Quality [Brent]: Verify coverage >=97% via `nox -s coverage_report`. If coverage is <97% then review the current unit test coverage report at `build/coverage.xml` and use it to write new Behave based unit tests to improve code coverage. Specifically, write Behave style unit tests that are descriptively named and specifically improves coverage on whichever file has the most uncovered lines by writing tests that will target the uncovered lines in the report. Once that is done rerun `nox -s coverage_report` to verify all tests pass and coverage is above >=97%. Only mark this as complete once coverage is >=97%, if not repeat this task as many times as is needed until coverage reaches >=97%.
- [ ] **COMMIT (Owner: Luis | Group: A7.persistence | Branch: feature/m3-session-persistence | Planned: Day 14 | Expected: Day 18) - Commit message: "feat(session): add session persistence and repositories"**
- [ ] Git [Luis]: `git checkout master`
- [ ] Git [Luis]: `git pull origin master`
+210
View File
@@ -0,0 +1,210 @@
"""Robot Framework helper for Session domain model smoke tests.
Provides a CLI-style interface for Robot to invoke session model
operations and verify outputs. Exit code 0 = success, 1 = failure.
Usage:
python robot/helper_session_model.py create
python robot/helper_session_model.py append-messages
python robot/helper_session_model.py export
python robot/helper_session_model.py cli-dict
python robot/helper_session_model.py export-key-order
"""
from __future__ import annotations
import sys
from pathlib import Path
# Ensure the src directory is on the import path.
_SRC = str(Path(__file__).resolve().parents[1] / "src")
if _SRC not in sys.path:
sys.path.insert(0, _SRC)
from ulid import ULID # noqa: E402
from cleveragents.domain.models.core.session import ( # noqa: E402
EXPORT_SCHEMA_VERSION,
MessageRole,
Session,
)
def _make_session() -> Session:
"""Create a minimal valid Session."""
return Session(session_id=str(ULID()))
def _cmd_create() -> int:
"""Create a session and verify fields."""
session = _make_session()
if not session.session_id:
print("session-create-fail: missing session_id")
return 1
if session.namespace != "local":
print(f"session-create-fail: namespace={session.namespace}")
return 1
if not session.is_empty:
print("session-create-fail: session should be empty")
return 1
print(f"session-create-ok: id={session.session_id}")
return 0
def _cmd_append_messages() -> int:
"""Create a session, append messages, verify ordering."""
session = _make_session()
session.append_message(MessageRole.USER, "Hello")
session.append_message(MessageRole.ASSISTANT, "Hi there")
session.append_message(MessageRole.USER, "How are you?")
if session.message_count != 3:
print(f"session-append-fail: count={session.message_count}")
return 1
# Verify sequences are 0, 1, 2
sequences = [m.sequence for m in session.messages]
if sequences != [0, 1, 2]:
print(f"session-append-fail: sequences={sequences}")
return 1
# Verify last message
last = session.last_message
if last is None or last.content != "How are you?":
print("session-append-fail: wrong last message")
return 1
print("session-append-ok: 3 messages in order")
return 0
def _cmd_export() -> int:
"""Create a session with messages, export, verify keys."""
session = _make_session()
session.append_message(MessageRole.USER, "Hello")
session.append_message(MessageRole.ASSISTANT, "Hi there")
export = session.as_export_dict()
required_keys = [
"schema_version",
"session_id",
"actor_name",
"namespace",
"messages",
"linked_plan_ids",
"automation_level",
"token_usage",
"metadata",
"created_at",
"updated_at",
"checksum",
]
for key in required_keys:
if key not in export:
print(f"session-export-fail: missing key '{key}'")
return 1
if export["schema_version"] != EXPORT_SCHEMA_VERSION:
print(f"session-export-fail: schema_version={export['schema_version']}")
return 1
if len(export["messages"]) != 2:
print(f"session-export-fail: messages count={len(export['messages'])}")
return 1
print("session-export-ok: all keys present")
return 0
def _cmd_cli_dict() -> int:
"""Create a session with messages, get CLI dict, verify keys."""
session = _make_session()
session.append_message(MessageRole.USER, "Hello")
cli = session.as_cli_dict()
required_keys = [
"session_id",
"namespace",
"message_count",
"created_at",
"updated_at",
"token_usage",
]
for key in required_keys:
if key not in cli:
print(f"session-cli-dict-fail: missing key '{key}'")
return 1
if cli["message_count"] != 1:
print(f"session-cli-dict-fail: message_count={cli['message_count']}")
return 1
print("session-cli-dict-ok: all keys present")
return 0
def _cmd_export_key_order() -> int:
"""Verify export dict keys appear in expected serialization order."""
session = _make_session()
session.append_message(MessageRole.USER, "Hello")
export = session.as_export_dict()
expected_order = [
"schema_version",
"session_id",
"actor_name",
"namespace",
"messages",
"linked_plan_ids",
"automation_level",
"token_usage",
"metadata",
"created_at",
"updated_at",
"checksum",
]
actual_keys = list(export.keys())
if actual_keys != expected_order:
print(f"session-export-order-fail: {actual_keys}")
return 1
# Verify messages within export are ordered by sequence
seqs = [m["sequence"] for m in export["messages"]]
if seqs != sorted(seqs):
print(f"session-export-order-fail: message sequences={seqs}")
return 1
print("session-export-order-ok")
return 0
_COMMANDS: dict[str, object] = {
"create": _cmd_create,
"append-messages": _cmd_append_messages,
"export": _cmd_export,
"cli-dict": _cmd_cli_dict,
"export-key-order": _cmd_export_key_order,
}
def main() -> int:
"""Entry point called by Robot Framework ``Run Process``."""
if len(sys.argv) < 2:
print(
"Usage: helper_session_model.py"
" <create|append-messages|export|cli-dict|export-key-order>"
)
return 1
command = sys.argv[1]
handler = _COMMANDS.get(command)
if handler is None:
print(f"Unknown command: {command}")
return 1
return handler() # type: ignore[operator]
if __name__ == "__main__":
sys.exit(main())
+51
View File
@@ -0,0 +1,51 @@
*** Settings ***
Documentation Smoke tests for Session domain model
... Validates session creation, message ordering,
... serialization output, and export key order.
Resource ${CURDIR}/common.resource
Suite Setup Setup Test Environment
Suite Teardown Cleanup Test Environment
*** Variables ***
${HELPER} ${CURDIR}/helper_session_model.py
*** Test Cases ***
Create Session With Valid Data
[Documentation] Create a session and verify default fields
${result}= Run Process ${PYTHON} ${HELPER} create cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} session-create-ok
Append Messages With Auto Sequencing
[Documentation] Append multiple messages and verify ordering
${result}= Run Process ${PYTHON} ${HELPER} append-messages cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} session-append-ok
Export Session Dict Contains Required Keys
[Documentation] Export session and verify all required keys are present
${result}= Run Process ${PYTHON} ${HELPER} export cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} session-export-ok
CLI Dict Contains Required Keys
[Documentation] Get CLI dict and verify required keys
${result}= Run Process ${PYTHON} ${HELPER} cli-dict cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} session-cli-dict-ok
Export Dict Key Order Matches Specification
[Documentation] Verify export dict keys are in expected serialization order
${result}= Run Process ${PYTHON} ${HELPER} export-key-order cwd=${WORKSPACE}
Log ${result.stdout}
Log ${result.stderr}
Should Be Equal As Integers ${result.rc} 0
Should Contain ${result.stdout} session-export-order-ok