forked from cleveragents/cleveragents-core
hurui200320
b6c3169634
Implements E2E test for devcontainer-driven development workflow.
Tests project setup with .devcontainer config, resource registration,
and full plan lifecycle with supervised automation profile.
Changes from Cycle 5 review:
1. Replaced custom WF16_ALLOW_XFAIL_SKIP escape hatch with the project
standard tdd_expected_fail tag system. Test is now tagged
[Tags] tdd_expected_fail tdd_bug tdd_bug_762 and the listener
inverts failures in CI until devcontainer features are fully wired.
2. Added diagnostic WF16 Test Teardown keyword that captures plan
status JSON on failure, mirroring the WF05 teardown pattern.
3. Added stderr checks to all Traceback/INTERNAL guard assertions
so tracebacks appearing only in stderr are also detected.
4. Rebased branch onto latest master (e4c01492).
5. Documented timeout budget: added comment explaining theoretical
worst-case (~46min) vs realistic execution under the 35min limit.
6. Added fail-fast warning in AC-3 check noting that AC-4, AC-5, and
AC-6 are dependent on AC-3 and are likely to also fail.
7. Reordered plan apply arguments to --yes --format json <plan_id>
for spec consistency.
8. Decomposed long ready_for_apply Evaluate expression into three
intermediate booleans (is_apply_phase, is_execute_complete,
is_already_applied) for readability.
9. Added shared_session.close() calls in resource_dag.robot before
each final print() to ensure explicit session cleanup.
10. Cleaned up stale xfail/skip classification references in log
messages.
Changes from earlier cycles (3-4) retained in this commit:
- Terminal state validation, AC output combination, JSON format flags,
suite-level init, Force Tags, automation profile fallback, ULID regex
IGNORECASE, AC-5 Evaluate decomposition, defensive re-check pattern.
ISSUES CLOSED: #762
120 lines
9.4 KiB
Plaintext
120 lines
9.4 KiB
Plaintext
*** Settings ***
|
|
Documentation Resource DAG Linking and Discovery Tests
|
|
Library Process
|
|
Library OperatingSystem
|
|
|
|
*** Variables ***
|
|
${PYTHON} python
|
|
|
|
*** Test Cases ***
|
|
Link Child And Verify Tree
|
|
[Documentation] Link a child resource and verify it appears in children list
|
|
${script}= Catenate SEPARATOR=\n
|
|
... import json
|
|
... from datetime import datetime, UTC
|
|
... from sqlalchemy import create_engine, event
|
|
... from sqlalchemy.orm import sessionmaker
|
|
... from cleveragents.infrastructure.database.models import Base
|
|
... from cleveragents.infrastructure.database.repositories import ResourceTypeRepository, ResourceRepository
|
|
... from cleveragents.domain.models.core.resource_type import ResourceTypeSpec, ResourceKind, SandboxStrategy
|
|
... from cleveragents.domain.models.core.resource import Resource, PhysVirt, ResourceCapabilities
|
|
... engine = create_engine("sqlite:///:memory:")
|
|
... @event.listens_for(engine, "connect")
|
|
... def _fk(conn, _): conn.cursor().execute("PRAGMA foreign_keys=ON")
|
|
... Base.metadata.create_all(engine)
|
|
... factory = sessionmaker(bind=engine)
|
|
... shared_session = factory()
|
|
... rt_repo = ResourceTypeRepository(lambda: shared_session)
|
|
... res_repo = ResourceRepository(lambda: shared_session)
|
|
... parent_spec = ResourceTypeSpec(name="robot/dag-parent", description="Parent", resource_kind=ResourceKind.PHYSICAL, sandbox_strategy=SandboxStrategy.NONE, user_addable=True, cli_args=[], parent_types=[], child_types=["robot/dag-child"], auto_discovery=None, equivalence=None, handler=None, capabilities={"read": True, "write": True, "sandbox": True, "checkpoint": False}, built_in=False)
|
|
... child_spec = ResourceTypeSpec(name="robot/dag-child", description="Child", resource_kind=ResourceKind.PHYSICAL, sandbox_strategy=SandboxStrategy.NONE, user_addable=True, cli_args=[], parent_types=[], child_types=[], auto_discovery=None, equivalence=None, handler=None, capabilities={"read": True, "write": True, "sandbox": True, "checkpoint": False}, built_in=False)
|
|
... rt_repo.create(parent_spec)
|
|
... rt_repo.create(child_spec)
|
|
... p = Resource(resource_id="01HDAGR0B0T0000000PARENT01", name=None, resource_type_name="robot/dag-parent", classification=PhysVirt.PHYSICAL, properties={}, location=None, capabilities=ResourceCapabilities(), created_at=datetime.now(tz=UTC), updated_at=datetime.now(tz=UTC))
|
|
... c = Resource(resource_id="01HDAGR0B0T00000CHXND00001", name=None, resource_type_name="robot/dag-child", classification=PhysVirt.PHYSICAL, properties={}, location=None, capabilities=ResourceCapabilities(), created_at=datetime.now(tz=UTC), updated_at=datetime.now(tz=UTC))
|
|
... res_repo.create(p)
|
|
... res_repo.create(c)
|
|
... res_repo.link_child("01HDAGR0B0T0000000PARENT01", "01HDAGR0B0T00000CHXND00001")
|
|
... children = res_repo.get_children("01HDAGR0B0T0000000PARENT01")
|
|
... assert len(children) == 1, f"Expected 1 child, got {len(children)}"
|
|
... assert children[0].resource_id == "01HDAGR0B0T00000CHXND00001"
|
|
... shared_session.close()
|
|
... print("Link child and verify tree passed")
|
|
${result}= Run Process ${PYTHON} -c ${script} timeout=60s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0 Link test failed: ${result.stderr}
|
|
Should Contain ${result.stdout} Link child and verify tree passed
|
|
|
|
Cycle Detection Rejects A To B To A
|
|
[Documentation] Linking A->B then B->A should raise CycleDetectedError
|
|
${script}= Catenate SEPARATOR=\n
|
|
... from datetime import datetime, UTC
|
|
... from sqlalchemy import create_engine, event
|
|
... from sqlalchemy.orm import sessionmaker
|
|
... from cleveragents.infrastructure.database.models import Base
|
|
... from cleveragents.infrastructure.database.repositories import ResourceTypeRepository, ResourceRepository, CycleDetectedError
|
|
... from cleveragents.domain.models.core.resource_type import ResourceTypeSpec, ResourceKind, SandboxStrategy
|
|
... from cleveragents.domain.models.core.resource import Resource, PhysVirt, ResourceCapabilities
|
|
... engine = create_engine("sqlite:///:memory:")
|
|
... @event.listens_for(engine, "connect")
|
|
... def _fk(conn, _): conn.cursor().execute("PRAGMA foreign_keys=ON")
|
|
... Base.metadata.create_all(engine)
|
|
... factory = sessionmaker(bind=engine)
|
|
... shared_session = factory()
|
|
... rt_repo = ResourceTypeRepository(lambda: shared_session)
|
|
... res_repo = ResourceRepository(lambda: shared_session)
|
|
... spec = ResourceTypeSpec(name="robot/cycle-type", description="Cycle", resource_kind=ResourceKind.PHYSICAL, sandbox_strategy=SandboxStrategy.NONE, user_addable=True, cli_args=[], parent_types=[], child_types=["robot/cycle-type"], auto_discovery=None, equivalence=None, handler=None, capabilities={"read": True, "write": True, "sandbox": True, "checkpoint": False}, built_in=False)
|
|
... rt_repo.create(spec)
|
|
... a = Resource(resource_id="01HDAGCYC000000000000000A1", name=None, resource_type_name="robot/cycle-type", classification=PhysVirt.PHYSICAL, properties={}, location=None, capabilities=ResourceCapabilities(), created_at=datetime.now(tz=UTC), updated_at=datetime.now(tz=UTC))
|
|
... b = Resource(resource_id="01HDAGCYC000000000000000B1", name=None, resource_type_name="robot/cycle-type", classification=PhysVirt.PHYSICAL, properties={}, location=None, capabilities=ResourceCapabilities(), created_at=datetime.now(tz=UTC), updated_at=datetime.now(tz=UTC))
|
|
... res_repo.create(a)
|
|
... res_repo.create(b)
|
|
... res_repo.link_child("01HDAGCYC000000000000000A1", "01HDAGCYC000000000000000B1")
|
|
... try:
|
|
... ${SPACE * 4}res_repo.link_child("01HDAGCYC000000000000000B1", "01HDAGCYC000000000000000A1")
|
|
... ${SPACE * 4}assert False, "Should have raised CycleDetectedError"
|
|
... except CycleDetectedError:
|
|
... ${SPACE * 4}print("Cycle detection passed")
|
|
... finally:
|
|
... ${SPACE * 4}shared_session.close()
|
|
${result}= Run Process ${PYTHON} -c ${script} timeout=60s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0 Cycle test failed: ${result.stderr}
|
|
Should Contain ${result.stdout} Cycle detection passed
|
|
|
|
Auto Discover Children
|
|
[Documentation] Auto-discover creates child resources per type rules
|
|
${script}= Catenate SEPARATOR=\n
|
|
... import json
|
|
... from datetime import datetime, UTC
|
|
... from sqlalchemy import create_engine, event
|
|
... from sqlalchemy.orm import sessionmaker
|
|
... from cleveragents.infrastructure.database.models import Base
|
|
... from cleveragents.infrastructure.database.repositories import ResourceTypeRepository, ResourceRepository
|
|
... from cleveragents.domain.models.core.resource_type import ResourceTypeSpec, ResourceKind, SandboxStrategy
|
|
... from cleveragents.domain.models.core.resource import Resource, PhysVirt, ResourceCapabilities
|
|
... engine = create_engine("sqlite:///:memory:")
|
|
... @event.listens_for(engine, "connect")
|
|
... def _fk(conn, _): conn.cursor().execute("PRAGMA foreign_keys=ON")
|
|
... Base.metadata.create_all(engine)
|
|
... factory = sessionmaker(bind=engine)
|
|
... shared_session = factory()
|
|
... rt_repo = ResourceTypeRepository(lambda: shared_session)
|
|
... res_repo = ResourceRepository(lambda: shared_session)
|
|
... parent_spec = ResourceTypeSpec(name="robot/disc-parent", description="Discoverer", resource_kind=ResourceKind.PHYSICAL, sandbox_strategy=SandboxStrategy.NONE, user_addable=True, cli_args=[], parent_types=[], child_types=["robot/disc-child"], auto_discovery={"enabled": True, "rules": [{"type": "robot/disc-child", "pattern": "*"}]}, equivalence=None, handler=None, capabilities={"read": True, "write": True, "sandbox": True, "checkpoint": False}, built_in=False)
|
|
... child_spec = ResourceTypeSpec(name="robot/disc-child", description="Discovered", resource_kind=ResourceKind.PHYSICAL, sandbox_strategy=SandboxStrategy.NONE, user_addable=True, cli_args=[], parent_types=[], child_types=[], auto_discovery=None, equivalence=None, handler=None, capabilities={"read": True, "write": True, "sandbox": True, "checkpoint": False}, built_in=False)
|
|
... rt_repo.create(parent_spec)
|
|
... rt_repo.create(child_spec)
|
|
... p = Resource(resource_id="01HDAGR0B0TDSC00PARENT0001", name=None, resource_type_name="robot/disc-parent", classification=PhysVirt.PHYSICAL, properties={}, location=None, capabilities=ResourceCapabilities(), created_at=datetime.now(tz=UTC), updated_at=datetime.now(tz=UTC))
|
|
... res_repo.create(p)
|
|
... created = res_repo.auto_discover_children("01HDAGR0B0TDSC00PARENT0001")
|
|
... assert len(created) >= 1, f"Expected >=1 children, got {len(created)}"
|
|
... assert created[0].resource_type_name == "robot/disc-child"
|
|
... children = res_repo.get_children("01HDAGR0B0TDSC00PARENT0001")
|
|
... assert len(children) >= 1, f"Expected >=1 linked children, got {len(children)}"
|
|
... shared_session.close()
|
|
... print("Auto discover children passed")
|
|
${result}= Run Process ${PYTHON} -c ${script} timeout=60s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0 Auto discover test failed: ${result.stderr}
|
|
Should Contain ${result.stdout} Auto discover children passed
|
|
|
|
*** Keywords ***
|