test(e2e): workflow example 14 — server mode team collaboration (supervised profile) #805
@@ -136,6 +136,11 @@
|
||||
non-serialisable config blobs (without exposing serializer internals in
|
||||
user-facing errors). Comprehensive BDD and Robot Framework
|
||||
tests cover all resolution paths and edge cases. (#901)
|
||||
- Fixed `list_actions()` to query the database when persistence is enabled
|
||||
so that actions created by previous CLI invocations are visible. Falls
|
||||
back to the in-memory cache on database errors or when no Unit of Work
|
||||
is wired. Added `ActionRepository.list_all()` for unfiltered action
|
||||
listing. (#760)
|
||||
- Added BuiltinAdapter class and MCP automatic resource slot creation.
|
||||
BuiltinAdapter wraps register_file_tools/register_git_tools/register_subplan_tool
|
||||
into a unified adapter interface. McpAdapter.infer_resource_slots() analyzes
|
||||
@@ -488,6 +493,12 @@
|
||||
Framework variables.
|
||||
(`robot/e2e/wf07_cicd.robot`, `robot/e2e/common_e2e.resource`,
|
||||
`robot/common_vars.py`) (#753)
|
||||
- Added E2E Robot Framework test for Specification Workflow Example 14: Server
|
||||
Mode — Team Collaboration. Exercises server mode configuration (server URL,
|
||||
token, namespace), diagnostics, action publishing to team namespace with
|
||||
namespace-scoped listing, actor registration, plan list smoke test, and
|
||||
`supervised` automation profile verification with threshold field assertions
|
||||
via real CLI with zero mocking. (`robot/e2e/wf14_server_mode.robot`) (#760)
|
||||
- Fixed `agents session list`, `agents session create`, and other session
|
||||
subcommands raising `AttributeError: 'DynamicContainer' object has no
|
||||
attribute 'db'` after `agents init`. Root cause: `_get_session_service()`
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
*** Settings ***
|
||||
Documentation E2E test for Workflow Example 14: server mode team collaboration.
|
||||
...
|
||||
... Validates the ``supervised`` automation profile in a distributed
|
||||
... team scenario where multiple engineers share actions, actors, and
|
||||
... projects via CleverAgents server mode. All commands exercise the
|
||||
... real CLI with zero mocking.
|
||||
...
|
||||
... Coverage: Steps 1–2 (server config, resource publishing).
|
||||
... Steps 3–4 (multi-machine usage, team monitoring) require a
|
||||
... live server and are deferred.
|
||||
Resource common_e2e.resource
|
||||
Suite Setup WF14 Suite Setup
|
||||
Suite Teardown WF14 Suite Teardown
|
||||
Force Tags E2E
|
||||
|
||||
*** Keywords ***
|
||||
WF14 Suite Setup
|
||||
[Documentation] E2E Suite Setup plus database initialisation for WF14 tests.
|
||||
E2E Suite Setup
|
||||
# Initialise the database so action/actor/plan commands work in all tests.
|
||||
${init}= Run CleverAgents Command init --yes
|
||||
Should Be Equal As Integers ${init.rc} 0
|
||||
# Generate a unique suffix for entity names to avoid UNIQUE
|
||||
# constraint collisions on repeated E2E runs or parallel CI.
|
||||
${suffix}= Evaluate __import__('uuid').uuid4().hex[:12]
|
||||
Set Suite Variable ${RUN_SUFFIX} ${suffix}
|
||||
|
||||
WF14 Suite Teardown
|
||||
[Documentation] Restore config state and clean up E2E test environment.
|
||||
# Reset config keys set during the server config test to avoid
|
||||
# contaminating subsequent test suites.
|
||||
Run Keyword And Ignore Error Run CleverAgents Command config set server.url ${EMPTY} expected_rc=None
|
||||
Run Keyword And Ignore Error Run CleverAgents Command config set server.token ${EMPTY} expected_rc=None
|
||||
Run Keyword And Ignore Error Run CleverAgents Command config set core.namespace local expected_rc=None
|
||||
E2E Suite Teardown
|
||||
|
||||
*** Test Cases ***
|
||||
WF14 E2E Server Config Setup
|
||||
[Documentation] Configure server URL, authentication token, and team namespace
|
||||
... for server mode per Specification Example 14 Step 1.
|
||||
# Set and verify server URL
|
||||
${set_url}= Run CleverAgents Command config set server.url https://agents.example.com
|
||||
Output Should Contain ${set_url} server.url
|
||||
${get_url}= Run CleverAgents Command config get server.url
|
||||
Output Should Contain ${get_url} https://agents.example.com
|
||||
# Set and verify server token (spec Step 1 includes token configuration)
|
||||
${set_token}= Run CleverAgents Command config set server.token tok_e2e_test_placeholder
|
||||
Output Should Contain ${set_token} server.token
|
||||
${get_token}= Run CleverAgents Command config get server.token
|
||||
Output Should Contain ${get_token} tok_e2e_test_placeholder
|
||||
# Set and verify team namespace
|
||||
${set_ns}= Run CleverAgents Command config set core.namespace myteam
|
||||
Output Should Contain ${set_ns} core.namespace
|
||||
${get_ns}= Run CleverAgents Command config get core.namespace
|
||||
Output Should Contain ${get_ns} myteam
|
||||
|
||||
WF14 E2E Diagnostics
|
||||
[Documentation] Run diagnostics and verify basic health-check output.
|
||||
... Diagnostics checks are all local (config, database, disk,
|
||||
... API keys, git, etc.) so the command should succeed even
|
||||
... without a live server. Server-mode diagnostics (Server
|
||||
... connectivity, Namespace membership) shown in the spec are
|
||||
... not yet implemented in the diagnostics command (spec gap).
|
||||
${result}= Run CleverAgents Command diagnostics --format plain expected_rc=0
|
||||
# Basic diagnostic categories should appear in the output
|
||||
${combined}= Set Variable ${result.stdout}\n${result.stderr}
|
||||
Should Contain Any ${combined} config Config configuration
|
||||
Should Contain Any ${combined} database Database db
|
||||
Should Contain Any ${combined} disk Disk space storage
|
||||
|
||||
WF14 E2E Action Create For Namespace
|
||||
[Documentation] Create an action in the team namespace and verify it persists.
|
||||
... Uses ``myteam/`` prefix per Specification Example 14 Step 2.
|
||||
${action_name}= Set Variable myteam/gen-tests-${RUN_SUFFIX}
|
||||
${yaml_content}= Catenate SEPARATOR=\n
|
||||
... name: ${action_name}
|
||||
... description: Generate unit tests for source modules
|
||||
... strategy_actor: local/strategist
|
||||
... execution_actor: local/executor
|
||||
... definition_of_done: All source modules have corresponding unit tests.
|
||||
... reusable: true
|
||||
... read_only: true
|
||||
${yaml_file}= Set Variable ${SUITE_HOME}${/}generate-tests-action.yaml
|
||||
Create File ${yaml_file} ${yaml_content}
|
||||
${create_result}= Run CleverAgents Command action create --config ${yaml_file}
|
||||
Output Should Contain ${create_result} gen-tests
|
||||
# Verify action persists across separate CLI invocations via action show
|
||||
${show_result}= Run CleverAgents Command action show ${action_name}
|
||||
Output Should Contain ${show_result} gen-tests
|
||||
# Make the action name available for the namespace listing test
|
||||
Set Suite Variable ${WF14_ACTION_NAME} ${action_name}
|
||||
|
||||
WF14 E2E Action List By Namespace
|
||||
[Documentation] List shared actions filtered by team namespace.
|
||||
... Validates Specification Example 14 Step 2:
|
||||
... ``agents action list --namespace myteam``.
|
||||
${list_result}= Run CleverAgents Command action list --namespace myteam --format plain
|
||||
Output Should Contain ${list_result} gen-tests
|
||||
|
||||
WF14 E2E Actor Add For Namespace
|
||||
[Documentation] Add an actor from YAML configuration and verify it appears.
|
||||
... Custom actors must use the ``local/`` namespace prefix per
|
||||
... the current ActorService validation.
|
||||
${actor_name}= Set Variable local/team-rev-${RUN_SUFFIX}
|
||||
${yaml_content}= Catenate SEPARATOR=\n
|
||||
... provider: openai
|
||||
... model: gpt-4
|
||||
... description: Team code reviewer actor for collaborative reviews
|
||||
${yaml_file}= Set Variable ${SUITE_HOME}${/}team-reviewer-actor.yaml
|
||||
Create File ${yaml_file} ${yaml_content}
|
||||
${add_result}= Run CleverAgents Command actor add ${actor_name} --config ${yaml_file}
|
||||
Output Should Contain ${add_result} team-rev
|
||||
# Use --format plain to avoid table truncation
|
||||
${list_result}= Run CleverAgents Command actor list --format plain
|
||||
Output Should Contain ${list_result} team-rev
|
||||
|
||||
WF14 E2E Plan List
|
||||
[Documentation] Verify that the plan list command executes without crashing.
|
||||
... No plans exist after init, so the output may be an empty table
|
||||
... or an informational message. The ``--namespace`` flag is not
|
||||
... yet implemented for plan list (spec gap).
|
||||
${result}= Run CleverAgents Command plan list expected_rc=0
|
||||
# After init, plan list should succeed (rc=0) even with an empty plan table
|
||||
Should Not Be Empty ${result.stdout}${result.stderr}
|
||||
|
||||
WF14 E2E Supervised Profile Verification
|
||||
[Documentation] Verify the built-in supervised automation profile exists and
|
||||
... contains the expected confidence threshold fields.
|
||||
${result}= Run CleverAgents Command automation-profile show supervised
|
||||
Output Should Contain ${result} supervised
|
||||
# Verify supervised-specific threshold fields are present in the output
|
||||
Output Should Contain ${result} auto_strategize
|
||||
Output Should Contain ${result} auto_execute
|
||||
Output Should Contain ${result} auto_apply
|
||||
@@ -622,8 +622,10 @@ class PlanLifecycleService:
|
||||
) -> list[Action]:
|
||||
"""List actions with optional filtering.
|
||||
|
||||
When persistence is enabled, results are merged from the in-memory
|
||||
cache and the database layer.
|
||||
When persistence is enabled, results are queried from the database
|
||||
so that actions created by previous CLI invocations are visible.
|
||||
Falls back to the in-memory cache on DB errors or when no UoW is
|
||||
wired.
|
||||
|
||||
Args:
|
||||
namespace: Filter by namespace
|
||||
@@ -632,6 +634,31 @@ class PlanLifecycleService:
|
||||
Returns:
|
||||
List of matching Actions
|
||||
"""
|
||||
# Query the database when persistence is enabled so that actions
|
||||
# created by previous CLI invocations are visible.
|
||||
if self._persisted and self.unit_of_work is not None:
|
||||
try:
|
||||
with self.unit_of_work.transaction() as ctx:
|
||||
if namespace:
|
||||
state_str = state.value if state else None
|
||||
actions = ctx.actions.get_by_namespace(
|
||||
namespace, state=state_str
|
||||
)
|
||||
else:
|
||||
actions = ctx.actions.list_all()
|
||||
# Refresh the in-memory cache with persisted actions.
|
||||
for a in actions:
|
||||
key = str(a.namespaced_name)
|
||||
if key not in self._actions:
|
||||
self._actions[key] = a
|
||||
# Apply state filter when list_all was used (no namespace).
|
||||
if state and not namespace:
|
||||
actions = [a for a in actions if a.state == state]
|
||||
return actions
|
||||
except DatabaseError:
|
||||
logger.debug("DB action list failed, falling back to in-memory cache")
|
||||
|
||||
# In-memory fallback (no persistence or DB error).
|
||||
actions = list(self._actions.values())
|
||||
|
||||
if namespace:
|
||||
|
||||
@@ -980,6 +980,27 @@ class ActionRepository:
|
||||
f"Failed to list actions in namespace '{namespace}': {exc}"
|
||||
) from exc
|
||||
|
||||
@database_retry
|
||||
def list_all(self) -> list[Any]:
|
||||
"""List all persisted actions, ordered by namespace then name.
|
||||
|
||||
Returns:
|
||||
List of ``Action`` domain objects.
|
||||
"""
|
||||
session = self._session()
|
||||
try:
|
||||
rows = (
|
||||
session.query(LifecycleActionModel)
|
||||
.order_by(
|
||||
LifecycleActionModel.namespace,
|
||||
LifecycleActionModel.name,
|
||||
)
|
||||
.all()
|
||||
)
|
||||
return [row.to_domain() for row in rows]
|
||||
except (OperationalError, SQLAlchemyDatabaseError) as exc:
|
||||
raise DatabaseError(f"Failed to list all actions: {exc}") from exc
|
||||
|
||||
@database_retry
|
||||
def get_by_state(self, state: str) -> list[Any]:
|
||||
"""List actions by state, ordered by ``updated_at`` DESC.
|
||||
|
||||
Reference in New Issue
Block a user