brent.edwards
e2c489aba2
CI / typecheck (push) Has been cancelled
CI / security (push) Has been cancelled
CI / benchmark-publish (push) Has been cancelled
CI / quality (push) Has been cancelled
CI / unit_tests (push) Has been cancelled
CI / integration_tests (push) Has been cancelled
CI / lint (push) Has been cancelled
CI / status-check (push) Has been cancelled
CI / build (push) Has been cancelled
CI / e2e_tests (push) Has been cancelled
CI / coverage (push) Has been cancelled
CI / benchmark-regression (push) Has been cancelled
CI / docker (push) Has been cancelled
CI / helm (push) Has been cancelled
Add Robot Framework integration test suite exercising Specification Workflow Example 6: documentation generation from codebase analysis using the trusted automation profile. The test suite (6 test cases) validates: - Context policy configuration with strategize/execute view-specific settings and view inheritance resolution - Budget enforcement filtering ContextFragments by max_file_size and max_total_size limits - Trusted profile auto-progress behavior (create_tool=0.0 triggers automatic Strategize→Execute; select_tool=1.0 gates Apply) - Action creation with doc_types/output_dir arguments and 3 invariants matching the spec scenario - Temp project sandbox with documentation generation into output directory and source-code modification invariant verification - Full plan lifecycle through trusted profile with argument and invariant preservation ISSUES CLOSED: #770 Co-authored-by: Brent E. Edwards <brent.edwards@cleverthis.com> Co-committed-by: Brent E. Edwards <brent.edwards@cleverthis.com>
66 lines
4.1 KiB
Plaintext
66 lines
4.1 KiB
Plaintext
*** Settings ***
|
|
Documentation Integration test: Workflow Example 6 — Documentation Generation
|
|
... from Codebase Analysis. Exercises the ``trusted`` automation
|
|
... profile with context policy configuration, code intelligence,
|
|
... and documentation-generation invariants using mocked LLM providers.
|
|
... Spec reference: docs/specification.md ~lines 38700-39039
|
|
... Issue: #770
|
|
Resource ${CURDIR}/common.resource
|
|
Suite Setup Setup Test Environment
|
|
Suite Teardown Cleanup Test Environment
|
|
|
|
*** Variables ***
|
|
${HELPER} robot/helper_int_wf06_doc_generation.py
|
|
|
|
*** Test Cases ***
|
|
Context Policy With View Specific Settings
|
|
[Documentation] Configure context policy with strategize and execute views
|
|
... and verify view inheritance resolution.
|
|
[Tags] wf06 context policy views
|
|
${result}= Run Process ${PYTHON} ${HELPER} context-policy-views cwd=${WORKSPACE} timeout=60s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0 Context policy test failed: ${result.stderr}
|
|
Should Contain ${result.stdout} context-policy-views-ok
|
|
|
|
Budget Enforcement Filters Fragments By Size Limits
|
|
[Documentation] Verify budget enforcement excludes fragments that exceed
|
|
... per-file and total size limits.
|
|
[Tags] wf06 context budget enforcement
|
|
${result}= Run Process ${PYTHON} ${HELPER} budget-enforcement cwd=${WORKSPACE} timeout=60s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0 Budget enforcement failed: ${result.stderr}
|
|
Should Contain ${result.stdout} budget-enforcement-ok
|
|
|
|
Trusted Profile Auto Progresses After Strategize
|
|
[Documentation] Verify the trusted profile auto-progresses after strategize
|
|
... (create_tool=0.0) but gates apply (select_tool=1.0).
|
|
[Tags] wf06 profile trusted lifecycle
|
|
${result}= Run Process ${PYTHON} ${HELPER} trusted-profile-behavior cwd=${WORKSPACE} timeout=60s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0 Trusted profile test failed: ${result.stderr}
|
|
Should Contain ${result.stdout} trusted-profile-behavior-ok
|
|
|
|
Action With Documentation Args And Invariants
|
|
[Documentation] Create generate-docs action with doc_types and output_dir
|
|
... args plus 3 source-code invariants, then use it to create
|
|
... a plan with the trusted profile.
|
|
[Tags] wf06 action args invariants
|
|
${result}= Run Process ${PYTHON} ${HELPER} action-with-doc-args cwd=${WORKSPACE} timeout=60s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0 Action creation failed: ${result.stderr}
|
|
Should Contain ${result.stdout} action-with-doc-args-ok
|
|
|
|
Doc Generation In Sandbox Preserves Source Invariant
|
|
[Documentation] Create temp project with source files, generate docs in
|
|
... sandbox output directory, verify documentation files exist
|
|
... and source files are unmodified.
|
|
[Tags] wf06 sandbox docs invariant source
|
|
${result}= Run Process ${PYTHON} ${HELPER} doc-generation-sandbox cwd=${WORKSPACE} timeout=60s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0 Doc generation sandbox failed: ${result.stderr}
|
|
Should Contain ${result.stdout} doc-generation-sandbox-ok
|
|
|
|
Full Trusted Lifecycle For Documentation Generation
|
|
[Documentation] End-to-end plan lifecycle with trusted profile: strategize
|
|
... auto-progresses, execute gates apply, explicit apply
|
|
... completes the plan with args and invariants preserved.
|
|
[Tags] wf06 lifecycle trusted full
|
|
${result}= Run Process ${PYTHON} ${HELPER} trusted-doc-lifecycle cwd=${WORKSPACE} timeout=60s on_timeout=kill
|
|
Should Be Equal As Integers ${result.rc} 0 Lifecycle test failed: ${result.stderr}
|
|
Should Contain ${result.stdout} trusted-doc-lifecycle-ok
|