[AUTO-UAT-POOL] UAT Testing Report (Cycle 2) #6263

New issue

Closed

opened 2026-04-09 19:37:03 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-09 19:37:03 +00:00

Owner

UAT Testing Pool Status — 2026-04-09

Agent: uat-tester (Pool Supervisor)
Instance ID: uat-pool-1
Cycle: 2
Master SHA: 666e86752f
Status: Starting

Feature Areas Identified

From docs/specification.md, the following feature areas will be tested:

Plan Lifecycle — Action→Strategize→Execute→Apply phases, states, transitions
CLI Commands - Plan — agents plan list/use/execute/apply/status/cancel/tree/explain/correct/diff/artifacts/prompt/rollback/errors
CLI Commands - Session — agents session create/list/show/delete/export/import/tell
CLI Commands - Project — agents project create/link-resource/unlink-resource/list/show/delete/context
CLI Commands - Actor — agents actor run/add/remove/list/show/context
CLI Commands - Resource — agents resource type/add/remove/list/show/inspect/tree/link-child/unlink-child/stop/rebuild
CLI Commands - Tool/Skill/Validation — agents tool/skill/validation add/remove/list/show
Invariants — Global/project/action/plan scoped, 4-tier precedence, non_overridable
Automation Profiles — 8 built-ins, custom profiles, safety profiles
A2A Protocol — Local stdio, server HTTP, extension methods _cleveragents/*
ACMS — UKO, CRP, context assembly pipeline, hot/warm/cold tiers
Actor System — YAML config, LLM composition, LangGraph, skills
Resource System — ULID identity, DAG organization, physical/virtual, devcontainer
Sandbox & Checkpoints — GitWorktree sandbox, checkpoint safety model
Correction Model — Decision tree editing, selective subtree recomputation
Output Formats — rich/color/table/plain/json/yaml for all commands

Dispatch Status

Workers being dispatched now...

Previously Found Bugs (Open)

#6260 agents plan artifacts JSON output issues
#6259 ValidationPipeline not wired into PlanLifecycleService
#6258 Advisory locking not integrated
#6257 agents plan diff JSON/YAML output issues
#6256 agents plan rollback JSON output wrong field names
#6254 Invariant merge missing ACTION scope
#6253 agents invariant add missing --non-overridable flag
#6237 agents action create drops fields from YAML
#6231 Custom automation profiles fall back to 'manual'
#6226 agents plan use --automation-profile rejects custom profiles
#6203 agents plan show command missing from CLI
#6201 A2aEventQueue not registered in DI container
#6161 ContainerToolExecutor never wired into application layer
#6160 agents project show JSON/YAML missing spec-required sections
#6155 agents plan correct --mode=revert not re-entering Strategize
#6154 agents actor run --skill silently ignores skill attachment
#6153 A2aLocalFacade legacy operations missing DeprecationWarning
#6151 agents diagnostics missing index health checks
#6149 agents plan correct --mode=append not persisting to decision tree
#6148 ChangeSetCapture not integrated with container execution
#6147 MCP adapter resource binding metadata lost
#6142 SEQUENTIAL_APPLY merge strategy semantically wrong
#6139 Several _cleveragents/* extension methods return stubs

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

# UAT Testing Pool Status — 2026-04-09 **Agent**: uat-tester (Pool Supervisor) **Instance ID**: uat-pool-1 **Cycle**: 2 **Master SHA**: 666e86752f26e329e1f0a9f9b8d346ea5142cc21 **Status**: Starting ## Feature Areas Identified From `docs/specification.md`, the following feature areas will be tested: 1. **Plan Lifecycle** — Action→Strategize→Execute→Apply phases, states, transitions 2. **CLI Commands - Plan** — `agents plan list/use/execute/apply/status/cancel/tree/explain/correct/diff/artifacts/prompt/rollback/errors` 3. **CLI Commands - Session** — `agents session create/list/show/delete/export/import/tell` 4. **CLI Commands - Project** — `agents project create/link-resource/unlink-resource/list/show/delete/context` 5. **CLI Commands - Actor** — `agents actor run/add/remove/list/show/context` 6. **CLI Commands - Resource** — `agents resource type/add/remove/list/show/inspect/tree/link-child/unlink-child/stop/rebuild` 7. **CLI Commands - Tool/Skill/Validation** — `agents tool/skill/validation add/remove/list/show` 8. **Invariants** — Global/project/action/plan scoped, 4-tier precedence, non_overridable 9. **Automation Profiles** — 8 built-ins, custom profiles, safety profiles 10. **A2A Protocol** — Local stdio, server HTTP, extension methods `_cleveragents/*` 11. **ACMS** — UKO, CRP, context assembly pipeline, hot/warm/cold tiers 12. **Actor System** — YAML config, LLM composition, LangGraph, skills 13. **Resource System** — ULID identity, DAG organization, physical/virtual, devcontainer 14. **Sandbox & Checkpoints** — GitWorktree sandbox, checkpoint safety model 15. **Correction Model** — Decision tree editing, selective subtree recomputation 16. **Output Formats** — rich/color/table/plain/json/yaml for all commands ## Dispatch Status Workers being dispatched now... ## Previously Found Bugs (Open) - #6260 `agents plan artifacts` JSON output issues - #6259 ValidationPipeline not wired into PlanLifecycleService - #6258 Advisory locking not integrated - #6257 `agents plan diff` JSON/YAML output issues - #6256 `agents plan rollback` JSON output wrong field names - #6254 Invariant merge missing ACTION scope - #6253 `agents invariant add` missing `--non-overridable` flag - #6237 `agents action create` drops fields from YAML - #6231 Custom automation profiles fall back to 'manual' - #6226 `agents plan use --automation-profile` rejects custom profiles - #6203 `agents plan show` command missing from CLI - #6201 A2aEventQueue not registered in DI container - #6161 ContainerToolExecutor never wired into application layer - #6160 `agents project show` JSON/YAML missing spec-required sections - #6155 `agents plan correct --mode=revert` not re-entering Strategize - #6154 `agents actor run --skill` silently ignores skill attachment - #6153 A2aLocalFacade legacy operations missing DeprecationWarning - #6151 `agents diagnostics` missing index health checks - #6149 `agents plan correct --mode=append` not persisting to decision tree - #6148 ChangeSetCapture not integrated with container execution - #6147 MCP adapter resource binding metadata lost - #6142 SEQUENTIAL_APPLY merge strategy semantically wrong - #6139 Several `_cleveragents/*` extension methods return stubs --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 commented

2026-04-09 19:52:42 +00:00

Author

Owner

Batch 1 Results — 8 Workers Completed

Timestamp: 2026-04-09
Workers dispatched: 8
Total bugs filed this batch: 13

Worker Results

Feature Area	Features Tested	Bugs Filed	Issue Numbers
Plan Lifecycle	10	1	#6289
CLI Session & Project	12	2	#6269, #6270
Invariants System	3	1	#6267
A2A Protocol	3	2	#6274, #6276
Actor System	2	2	#6283, #6286
Resource System	9	1	#6291
ACMS Context System	10	4	#6282, #6284, #6285, #6287
Tool/Skill/Validation	4	0	(none new)

New Bugs Filed This Batch

#6289 — subplan_parallel_spawn decisions run sequentially instead of concurrently
#6269 — Session CLI issue (details in issue)
#6270 — Project CLI issue (details in issue)
#6267 — Invariants CLI does not honor documented --scope/--text interface
#6274 — CLI calls application services directly, bypassing A2A facade
#6276 — A2A server mode HTTP transport always raises A2aNotAvailableError
#6283 — agents actor add/run never parse v3 YAML schema (LangGraph routes, skills, LSP bindings ignored)
#6286 — agents actor context missing list/show/clear subcommands
#6291 — Devcontainer auto-discovery never materializes devcontainer-instance children
#6282 — Skeleton propagation never triggered
#6284 — ContextTierService never hydrated during plan execution
#6285 — Projects lack required indexing_status field
#6287 — ACMS indexing pipeline (UKOIndexer) not wired into CLI commands

Remaining Feature Areas for Batch 2

Automation Profiles (8 built-ins, custom profiles, safety profiles)
Sandbox & Checkpoints (GitWorktree sandbox, checkpoint safety model)
Correction Model (decision tree editing, selective subtree recomputation)
Output Formats (rich/color/table/plain/json/yaml for all commands)
CLI Commands - Action (create/list/show/archive)
LSP Integration (agents lsp add/remove/list/show/serve)
Config Commands (agents config set/get/list)
Diagnostics & Version (agents diagnostics, agents version, agents info)

Dispatching Batch 2 now...

Automated by CleverAgents Bot
Supervisor: UAT Testing | Agent: uat-tester

## Batch 1 Results — 8 Workers Completed **Timestamp**: 2026-04-09 **Workers dispatched**: 8 **Total bugs filed this batch**: 13 ### Worker Results | Feature Area | Features Tested | Bugs Filed | Issue Numbers | |---|---|---|---| | Plan Lifecycle | 10 | 1 | #6289 | | CLI Session & Project | 12 | 2 | #6269, #6270 | | Invariants System | 3 | 1 | #6267 | | A2A Protocol | 3 | 2 | #6274, #6276 | | Actor System | 2 | 2 | #6283, #6286 | | Resource System | 9 | 1 | #6291 | | ACMS Context System | 10 | 4 | #6282, #6284, #6285, #6287 | | Tool/Skill/Validation | 4 | 0 | (none new) | ### New Bugs Filed This Batch - **#6289** — `subplan_parallel_spawn` decisions run sequentially instead of concurrently - **#6269** — Session CLI issue (details in issue) - **#6270** — Project CLI issue (details in issue) - **#6267** — Invariants CLI does not honor documented `--scope`/`--text` interface - **#6274** — CLI calls application services directly, bypassing A2A facade - **#6276** — A2A server mode HTTP transport always raises `A2aNotAvailableError` - **#6283** — `agents actor add/run` never parse v3 YAML schema (LangGraph routes, skills, LSP bindings ignored) - **#6286** — `agents actor context` missing `list/show/clear` subcommands - **#6291** — Devcontainer auto-discovery never materializes `devcontainer-instance` children - **#6282** — Skeleton propagation never triggered - **#6284** — `ContextTierService` never hydrated during plan execution - **#6285** — Projects lack required `indexing_status` field - **#6287** — ACMS indexing pipeline (`UKOIndexer`) not wired into CLI commands ### Remaining Feature Areas for Batch 2 - Automation Profiles (8 built-ins, custom profiles, safety profiles) - Sandbox & Checkpoints (GitWorktree sandbox, checkpoint safety model) - Correction Model (decision tree editing, selective subtree recomputation) - Output Formats (rich/color/table/plain/json/yaml for all commands) - CLI Commands - Action (create/list/show/archive) - LSP Integration (agents lsp add/remove/list/show/serve) - Config Commands (agents config set/get/list) - Diagnostics & Version (agents diagnostics, agents version, agents info) Dispatching Batch 2 now... --- **Automated by CleverAgents Bot** Supervisor: UAT Testing | Agent: uat-tester

HAL9000 added the

labels

2026-04-09 20:20:40 +00:00

HAL9000 closed this issue

2026-04-10 23:35:10 +00:00