Files
cleveragents-core/features/wf04_snapshot_helper.feature
T
freemo 845cf61b47
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 38s
CI / security (pull_request) Successful in 1m17s
CI / build (pull_request) Successful in 25s
CI / helm (pull_request) Successful in 37s
CI / quality (pull_request) Successful in 3m43s
CI / typecheck (pull_request) Successful in 3m58s
CI / integration_tests (pull_request) Successful in 6m4s
CI / unit_tests (pull_request) Successful in 6m22s
CI / docker (pull_request) Successful in 21s
CI / e2e_tests (pull_request) Successful in 9m29s
CI / coverage (pull_request) Successful in 9m12s
CI / status-check (pull_request) Successful in 1s
CI / lint (push) Successful in 31s
CI / security (push) Successful in 55s
CI / build (push) Successful in 21s
CI / helm (push) Successful in 22s
CI / quality (push) Successful in 3m45s
CI / typecheck (push) Successful in 3m56s
CI / benchmark-regression (push) Has been skipped
CI / unit_tests (push) Successful in 6m46s
CI / integration_tests (push) Successful in 6m25s
CI / docker (push) Successful in 1m20s
CI / e2e_tests (push) Successful in 9m57s
CI / coverage (push) Successful in 10m50s
CI / status-check (push) Successful in 1s
CI / benchmark-publish (push) Successful in 29m50s
CI / benchmark-regression (pull_request) Successful in 56m29s
test(e2e): workflow example 4 — multi-project dependency update (supervised profile)
Implement WF04 E2E test coverage for multi-project dependency update
using the supervised automation profile. A parent plan targets four
projects (common-lib + 3 services), spawns child subplans per project,
executes in dependency order, validates per project, and applies in
dependency order.

Key components:
- Robot test (robot/e2e/wf04_multi_project.robot): full supervised
  workflow exercising plan use, strategize, execute, validate, apply,
  and final status. JSON output parsing via raw_decode for resilience.
  Test-level guard skips the entire test (visible SKIPPED in CI) when
  the LLM produces zero subplans, preventing silent AC-4/5/6/7 bypass.
  Count Decision Nodes keyword delegates to the Python helper to avoid
  duplicating recursive tree-walking logic.
- Snapshot helper (robot/e2e/wf04_snapshot_helper.py): collects
  deterministic parent/subplan metadata from the lifecycle service.
  Timestamps normalised to UTC. Depth-guarded count_decision_nodes()
  function (max_depth=50) for safe reuse from Robot keywords.
  sys.path.append (not insert) to avoid shadowing the standard library.
- Behave unit tests (features/wf04_snapshot_helper.feature + steps):
  17 scenarios covering _iso(), _enum_value(), count_decision_nodes(),
  and _build_snapshot() with mocked lifecycle service. Includes tests
  for no-subplans (verifies plan_id, project_scopes, validation_summary),
  with-subplans (verifies concrete mapped project names and subplan_count),
  child-plan-is-None (verifies empty-string defaults), unmapped resources,
  and exact-match aware-datetime UTC normalisation.

ISSUES CLOSED: #750
2026-03-30 11:47:37 +08:00

142 lines
5.6 KiB
Gherkin

@phase1 @e2e_helpers @wf04
Feature: WF04 Snapshot Helper Utilities
As a developer maintaining the WF04 E2E test infrastructure
I want unit tests for the snapshot helper's utility functions
So that timestamp normalisation, enum serialisation, decision counting,
and snapshot building are verified without running a full E2E test
# --- _iso() timestamp normalisation ---
@iso_timestamp
Scenario: _iso returns empty string for None
Given a None timestamp value
When _iso is called
Then the iso result should be an empty string
@iso_timestamp
Scenario: _iso returns empty string for non-datetime truthy value
Given a non-datetime truthy value "not-a-date"
When _iso is called
Then the iso result should be an empty string
@iso_timestamp
Scenario: _iso normalises naive datetime to UTC
Given a naive datetime 2026-03-15T10:30:00
When _iso is called
Then the iso result should be exactly "2026-03-15T10:30:00+00:00"
@iso_timestamp
Scenario: _iso normalises aware datetime to UTC
Given an aware datetime 2026-03-15T10:30:00 in timezone UTC+05:00
When _iso is called
Then the iso result should be exactly "2026-03-15T05:30:00+00:00"
@iso_timestamp
Scenario: _iso produces consistent format for mixed timezone inputs
Given a naive datetime 2026-03-15T12:00:00
And an aware datetime 2026-03-15T12:00:00 in timezone UTC
When _iso is called on both
Then both iso results should be identical
# --- _enum_value() serialisation ---
@enum_value
Scenario: _enum_value returns .value for enum-like objects
Given an enum-like object with value "completed"
When _enum_value is called
Then the enum result should be "completed"
@enum_value
Scenario: _enum_value returns str for plain objects
Given a plain string "hello"
When _enum_value is called
Then the enum result should be "hello"
# --- count_decision_nodes() ---
@decision_count
Scenario: count_decision_nodes returns 0 for empty input
Given an empty list tree
When count_decision_nodes is called
Then the decision node count should be 0
@decision_count
Scenario: count_decision_nodes counts single root node
Given a tree with one decision node
When count_decision_nodes is called
Then the decision node count should be 1
@decision_count
Scenario: count_decision_nodes counts nested children
Given a tree with a root and 2 child decision nodes
When count_decision_nodes is called
Then the decision node count should be 3
@decision_count
Scenario: count_decision_nodes skips nodes without decision_id
Given a tree with mixed decision and non-decision nodes
When count_decision_nodes is called
Then the decision node count should be 2
@decision_count
Scenario: count_decision_nodes handles list input
Given a list of 2 independent decision trees
When count_decision_nodes is called
Then the decision node count should be 2
@decision_count
Scenario: count_decision_nodes truncates at max_depth
Given a tree with depth 5 and one decision node per level
When count_decision_nodes is called with max_depth 3
Then the decision node count should be 3
# --- _build_snapshot() with mocked lifecycle service ---
@snapshot_build
Scenario: _build_snapshot raises ValueError for missing plan
Given a mocked lifecycle service that returns None for plan lookup
When _build_snapshot is called with plan_id "nonexistent"
Then the snapshot error should be ValueError containing "not found"
@snapshot_build
Scenario: _build_snapshot returns correct structure for plan without subplans
Given a mocked lifecycle service with a plan that has no subplans
When _build_snapshot is called
Then the snapshot should have subplan_count 0
And the snapshot subplans list should be empty
And the snapshot plan_id should match the mocked plan ID
And the snapshot project_scopes should be an empty list
And the snapshot validation_summary should be None
@snapshot_build
Scenario: _build_snapshot maps resources to projects for subplans
Given a mocked lifecycle service with a plan that has 2 subplans and project scopes
When _build_snapshot is called
Then the snapshot should have subplan_count 2
And subplan "SUB01" should map to project "proj-a"
And subplan "SUB02" should map to project "proj-b"
And each subplan should have mapped_projects populated
And each subplan should have an unmapped_resources field
And subplan "SUB01" status should be "completed"
And subplan "SUB01" child_phase should be "completed"
And subplan "SUB01" started_at should be non-empty
And subplan "SUB01" child_validation_summary required_passed should be 1
@snapshot_build
Scenario: _build_snapshot includes unmapped_resources for unknown resource IDs
Given a mocked lifecycle service with a subplan targeting an unmapped resource
When _build_snapshot is called
Then the subplan unmapped_resources should contain the unknown resource ID
@snapshot_build
Scenario: _build_snapshot defaults child fields when child plan is None
Given a mocked lifecycle service with a subplan whose child plan is None
When _build_snapshot is called
Then the subplan child_phase should be an empty string
And the subplan child_state should be an empty string
And the subplan child_updated_at should be an empty string
And the subplan execute_started_at should be an empty string
And the subplan execute_completed_at should be an empty string
And the subplan apply_started_at should be an empty string
And the subplan applied_at should be an empty string