freemo
aa008ba5d7
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 15s
CI / quality (pull_request) Successful in 18s
CI / build (pull_request) Successful in 17s
CI / typecheck (pull_request) Successful in 30s
CI / security (pull_request) Successful in 32s
CI / integration_tests (pull_request) Successful in 4m2s
CI / unit_tests (pull_request) Successful in 7m26s
CI / docker (pull_request) Successful in 9s
CI / benchmark-regression (pull_request) Successful in 16m11s
CI / coverage (pull_request) Successful in 23m7s
CI / lint (push) Successful in 26s
CI / quality (push) Successful in 19s
CI / typecheck (push) Successful in 1m4s
CI / build (push) Successful in 39s
CI / security (push) Successful in 1m2s
CI / benchmark-regression (push) Has been skipped
CI / integration_tests (push) Successful in 4m15s
CI / benchmark-publish (push) Successful in 11m19s
CI / unit_tests (push) Successful in 14m6s
CI / docker (push) Successful in 9s
CI / coverage (push) Successful in 37m7s
Add extended CLI flags and output rendering for plan and action commands in the v3 plan lifecycle. Plan use command: - --automation-profile flag validates against BUILTIN_PROFILES and persists to plan metadata with PLAN provenance - --invariant flag (repeatable) passes PlanInvariant objects with InvariantSource.PLAN to PlanLifecycleService.use_action() - --strategy-actor, --execution-actor, --estimation-actor, and --invariant-actor flags validate namespace/name format via validate_namespaced_actor() before setting on the plan Plan status/list output: - Profile and Invariants columns added to plan status summary table - Profile and Invariants columns added to lifecycle-list table - _plan_spec_dict includes estimation_actor, invariant_actor, and invariants fields in all output formats (json, yaml, plain, table) - _print_lifecycle_plan rich panel displays invariants with source provenance tags Action show output: - _print_action displays estimation_actor, invariant_actor, invariants list, inputs_schema, and automation_profile when present - _action_spec_dict conditionally includes estimation_actor, invariant_actor, and inputs_schema fields Documentation: - docs/reference/plan_cli.md documents all extended flags, actor override validation, and usage examples - docs/reference/action_cli.md documents extended output fields and YAML configuration options Tests: - 29 Behave scenarios in features/cli_extensions.feature covering automation profile, invariant flags, actor override validation, plan status/list rendering, and action show extended fields - 5 Robot Framework integration tests in robot/cli_extensions.robot for plan use with invariants, profiles, and actor validation - 6 ASV benchmark suites in benchmarks/cli_extensions_bench.py measuring parsing overhead for all new flags and rendering paths ISSUES CLOSED: #325
192 lines
8.9 KiB
Gherkin
192 lines
8.9 KiB
Gherkin
Feature: CLI extensions for plan and action commands
|
|
As a developer using the CleverAgents CLI
|
|
I want automation profile, invariant, and actor override flags
|
|
So that I can customize plan creation and view extended details
|
|
|
|
Background:
|
|
Given a cli extensions test runner
|
|
And a cli extensions mocked lifecycle service
|
|
|
|
# --- plan use: automation profile ---
|
|
|
|
Scenario: Plan use with --automation-profile persists to plan metadata
|
|
Given a cli extensions action exists
|
|
When I run plan use with automation profile flag "trusted"
|
|
Then the cli extensions plan use should succeed
|
|
And the cli extensions plan should have automation profile "trusted"
|
|
|
|
Scenario: Plan use with invalid automation profile is rejected
|
|
Given a cli extensions action exists
|
|
When I run plan use with automation profile flag "nonexistent-profile"
|
|
Then the cli extensions plan use should fail
|
|
|
|
# --- plan use: invariant flags ---
|
|
|
|
Scenario: Plan use with --invariant flags persists to plan metadata
|
|
Given a cli extensions action exists
|
|
When I run plan use with invariant flags "No warnings" and "Keep compat"
|
|
Then the cli extensions plan use should succeed
|
|
And the cli extensions service received invariants "No warnings" and "Keep compat"
|
|
|
|
Scenario: Plan use with single invariant flag
|
|
Given a cli extensions action exists
|
|
When I run plan use with single invariant "All tests must pass"
|
|
Then the cli extensions plan use should succeed
|
|
And the cli extensions service received single invariant "All tests must pass"
|
|
|
|
# --- plan use: actor override validation (valid namespaced) ---
|
|
|
|
Scenario: Plan use with valid namespaced strategy-actor
|
|
Given a cli extensions action exists
|
|
When I run plan use with strategy actor flag "openai/gpt-4"
|
|
Then the cli extensions plan use should succeed
|
|
And the cli extensions plan has strategy actor "openai/gpt-4"
|
|
|
|
Scenario: Plan use with valid namespaced execution-actor
|
|
Given a cli extensions action exists
|
|
When I run plan use with execution actor flag "anthropic/claude-3"
|
|
Then the cli extensions plan use should succeed
|
|
And the cli extensions plan has execution actor "anthropic/claude-3"
|
|
|
|
Scenario: Plan use with valid namespaced estimation-actor
|
|
Given a cli extensions action exists
|
|
When I run plan use with estimation actor flag "openai/gpt-4"
|
|
Then the cli extensions plan use should succeed
|
|
And the cli extensions plan has estimation actor "openai/gpt-4"
|
|
|
|
Scenario: Plan use with valid namespaced invariant-actor
|
|
Given a cli extensions action exists
|
|
When I run plan use with invariant actor flag "openai/gpt-4"
|
|
Then the cli extensions plan use should succeed
|
|
And the cli extensions plan has invariant actor "openai/gpt-4"
|
|
|
|
# --- plan use: actor override validation (invalid format) ---
|
|
|
|
Scenario: Plan use rejects invalid strategy-actor format
|
|
Given a cli extensions action exists
|
|
When I run plan use with strategy actor flag "bad-format"
|
|
Then the cli extensions plan use should fail with actor validation error
|
|
|
|
Scenario: Plan use rejects invalid execution-actor format
|
|
Given a cli extensions action exists
|
|
When I run plan use with execution actor flag "no_slash"
|
|
Then the cli extensions plan use should fail with actor validation error
|
|
|
|
Scenario: Plan use rejects invalid estimation-actor format
|
|
Given a cli extensions action exists
|
|
When I run plan use with estimation actor flag "UPPERCASE/gpt"
|
|
Then the cli extensions plan use should fail with actor validation error
|
|
|
|
Scenario: Plan use rejects invalid invariant-actor format
|
|
Given a cli extensions action exists
|
|
When I run plan use with invariant actor flag "spaces are bad"
|
|
Then the cli extensions plan use should fail with actor validation error
|
|
|
|
# --- plan status: showing automation_profile ---
|
|
|
|
Scenario: Plan status shows automation profile in table
|
|
Given cli extensions plans exist with automation profile "trusted"
|
|
When I run cli extensions plan status
|
|
Then the cli extensions status output should contain "trusted"
|
|
|
|
Scenario: Plan status shows automation profile in JSON
|
|
Given cli extensions plans exist with automation profile "manual"
|
|
When I run cli extensions plan status with format "json"
|
|
Then the cli extensions status json output should contain automation profile "manual"
|
|
|
|
# --- plan list: showing invariants when present ---
|
|
|
|
Scenario: Plan lifecycle-list shows invariant count in table
|
|
Given cli extensions plans exist with invariants
|
|
When I run cli extensions plan lifecycle-list
|
|
Then the cli extensions list output should contain invariant count
|
|
|
|
Scenario: Plan lifecycle-list shows invariants in JSON
|
|
Given cli extensions plans exist with invariants
|
|
When I run cli extensions plan lifecycle-list with format "json"
|
|
Then the cli extensions list json output should contain invariants
|
|
|
|
# --- action show: optional actors ---
|
|
|
|
Scenario: Action show displays estimation actor when present
|
|
Given a cli extensions action with estimation actor "openai/gpt-4"
|
|
When I run cli extensions action show
|
|
Then the cli extensions action output should contain "Estimation Actor"
|
|
And the cli extensions action output should contain "openai/gpt-4"
|
|
|
|
Scenario: Action show displays invariant actor when present
|
|
Given a cli extensions action with invariant actor "anthropic/claude-3"
|
|
When I run cli extensions action show
|
|
Then the cli extensions action output should contain "Invariant Actor"
|
|
And the cli extensions action output should contain "anthropic/claude-3"
|
|
|
|
Scenario: Action show displays invariants when present
|
|
Given a cli extensions action with invariants
|
|
When I run cli extensions action show
|
|
Then the cli extensions action output should contain "Invariants"
|
|
|
|
Scenario: Action show displays inputs schema when present
|
|
Given a cli extensions action with inputs schema
|
|
When I run cli extensions action show
|
|
Then the cli extensions action output should contain "Inputs Schema"
|
|
|
|
Scenario: Action show in JSON includes optional actors
|
|
Given a cli extensions action with estimation actor "openai/gpt-4"
|
|
When I run cli extensions action show with format "json"
|
|
Then the cli extensions action json should contain "estimation_actor"
|
|
|
|
Scenario: Action show in JSON includes inputs schema
|
|
Given a cli extensions action with inputs schema
|
|
When I run cli extensions action show with format "json"
|
|
Then the cli extensions action json should contain "inputs_schema"
|
|
|
|
# --- plan use: automation_profile + invariants combined ---
|
|
|
|
Scenario: Plan use with automation profile and invariants together
|
|
Given a cli extensions action exists
|
|
When I run plan use with profile "trusted" and invariant "No regressions"
|
|
Then the cli extensions plan use should succeed
|
|
And the cli extensions plan should have automation profile "trusted"
|
|
And the cli extensions service received single invariant "No regressions"
|
|
|
|
# --- plan status: single plan view with invariants ---
|
|
|
|
Scenario: Plan status single plan view renders invariants in rich panel
|
|
Given cli extensions single plan exists with invariants
|
|
When I run cli extensions plan status for single plan
|
|
Then the cli extensions status output should contain "Invariants"
|
|
And the cli extensions status output should contain "No warnings"
|
|
And the cli extensions status output should contain "plan"
|
|
|
|
Scenario: Plan status single plan view renders automation profile in rich panel
|
|
Given cli extensions single plan exists with profile "trusted"
|
|
When I run cli extensions plan status for single plan
|
|
Then the cli extensions status output should contain "Automation Profile"
|
|
And the cli extensions status output should contain "trusted"
|
|
|
|
# --- action show: automation profile display ---
|
|
|
|
Scenario: Action show displays automation profile when present
|
|
Given a cli extensions action with automation profile "trusted"
|
|
When I run cli extensions action show
|
|
Then the cli extensions action output should contain "Automation Profile"
|
|
And the cli extensions action output should contain "trusted"
|
|
|
|
Scenario: Action show in JSON without invariant actor omits field
|
|
Given a cli extensions action without invariant actor
|
|
When I run cli extensions action show with format "json"
|
|
Then the cli extensions action json should not contain "invariant_actor"
|
|
|
|
# --- validate_namespaced_actor function ---
|
|
|
|
Scenario: validate_namespaced_actor accepts valid names
|
|
Then validate_namespaced_actor accepts "openai/gpt-4"
|
|
And validate_namespaced_actor accepts "anthropic/claude-3"
|
|
And validate_namespaced_actor accepts "local/my-actor"
|
|
|
|
Scenario: validate_namespaced_actor rejects invalid names
|
|
Then validate_namespaced_actor rejects "no-slash"
|
|
And validate_namespaced_actor rejects "UPPER/case"
|
|
And validate_namespaced_actor rejects "/leading-slash"
|
|
And validate_namespaced_actor rejects "trail/ing/"
|