TDD expected-fail tests proving bug #822 exists:
CheckpointService.rollback_to_checkpoint() returns a successful
RollbackResult but does not execute git reset --hard. Files modified
after the checkpoint remain unchanged after rollback.
Also fixes Robot Framework timeout robustness across the entire test
suite: all Run Process calls now use on_timeout=kill (prevents
SIGTERM-induced -15 exit codes under CI load) and timeouts increased
to 120s (prevents premature kills during heavy parallel execution).
ISSUES CLOSED: #839
Replace the tolerant "rc == 0 or rc == 1" assertion with a strict
"rc == 0" check for the diagnostics --check command. In a clean test
environment, diagnostics should complete without finding errors, and the
previous assertion could not distinguish a legitimate diagnostic failure
from a command crash.
Add LLMTrace Pydantic v2 domain model with all required fields (trace_id,
plan_id, decision_id, actor, provider, model, prompt_tokens, completion_tokens,
cost_usd, latency_ms, tool_calls, context_hash, streaming, retry_count, error).
Define 14 OperationalMetricKey values (PLAN_DURATION_MS, PLAN_TOTAL_COST_USD,
PLAN_DECISION_COUNT, ACTOR_INVOCATION_COUNT, ACTOR_LATENCY_MS,
TOOL_INVOCATION_COUNT, TOOL_ERROR_RATE, CONTEXT_BUILD_TIME_MS,
CONTEXT_TOKEN_COUNT, LLM_CALL_COUNT, LLM_TOTAL_TOKENS, LLM_TOTAL_COST_USD,
LLM_AVG_LATENCY_MS, SUBPLAN_COUNT) with MetricEntry model and MetricCollector.
Add llm_traces database table (LLMTraceModel) with LLMTraceRepository for
persistence. TraceService provides recording, querying, metric computation,
plan lifecycle hooks, and optional LangSmith forwarding when
LANGCHAIN_TRACING_V2=true.
Wired into DI container as trace_service. Includes 28 Behave BDD scenarios,
6 Robot Framework smoke tests, 3 ASV benchmark suites, and reference
documentation.
Fix pre-existing cli_core server_mode test flake by mocking
resolve_server_mode in test steps to avoid stale config file interference.
Closes#500
Add domain models, application service, CLI integration, and full test
coverage for multi-project subplan orchestration.
New components:
- MultiProjectMetadata, ProjectScope, ProjectDependency domain models
- MultiProjectService for creating/managing multi-project plans
- CLI display of multi-project metadata in plan commands
- Behave BDD tests (20 scenarios), Robot Framework integration tests,
ASV benchmarks, and reference documentation
Also fixes pre-existing test flakiness in cli_core, core_cli_commands,
cli_plan_context_commands, and helper_server_stubs caused by environment
leakage and path-with-spaces issues.
ISSUES CLOSED: #199
- Add timeout=60s to all Run Process calls in cli.robot (previously had
no timeout, causing hangs under CI parallel load)
- Increase cli_core.robot timeouts from 30s to 60s to match other robot
files
- Relax performance assertions from <30s to <60s to account for Python
cold-start overhead under pabot parallel execution
The test now accepts both exit code 0 (no errors) and 1 (has errors)
since CI environments may have low disk space or missing tools that
trigger error-level diagnostic checks.
Add structured data builders and Rich renderers for version, info, and
diagnostics commands. Support --format rich/plain/json/yaml output parity
and --check flag for diagnostics (exits non-zero on errors).
Includes Behave scenarios, Robot smoke tests, ASV benchmarks, and updates
existing tests to match the new Rich panel output format.
brent.edwards
freemo
khyari hamza