Add multi-level tests for TUI shell safety guards #9667

New issue

Open

opened 2026-04-15 01:38:18 +00:00 by HAL9000 · 1 comment

HAL9000 commented

2026-04-15 01:38:18 +00:00

Owner

Summary

src/cleveragents/tui/shell_safety/*.py implements the danger detector for ! shell execution, yet the package has no Behave features, Robot suites, or ASV benchmarks.
Regression bugs such as #8466 and UAT #7964 keep resurfacing because the detector and warning objects are untested.
Multi-level tests are required to keep the spec danger classifications stable.

Evidence

Repository tree queries show zero feature files mentioning shell_safety.
Robot suite listings likewise contain no references to the module.
The benchmarks directory has no entries for shell safety, so detector performance is untracked.
DangerousPatternDetector, pattern_registry.DEFAULT_PATTERNS, and DangerousCommandWarning guard shell execution without automated coverage.

Proposed Tests

Behave: add features/tui_shell_safety.feature covering detector initialisation, add/remove/replace operations, CRITICAL/HIGH/MEDIUM/LOW classification, warning formatting, and allowlist/logging toggles.
Robot: introduce a TUI integration flow that runs with CLEVERAGENTS_ALLOW_DANGEROUS_SHELL=1, submits commands across danger levels through the live input router, and verifies that dangerous commands surface modal warnings while safe commands bypass them.
ASV: benchmark detector throughput (check/check_first over representative batches) and registry mutation costs so pattern-catalog growth has performance alarms.

Duplicate Check

GET /api/v1/repos/cleveragents/cleveragents-core/issues?state=open&limit=50&page=1..112 filtered for shell safety terms (reviewed #8466, #7964, #6361 — behaviour fixes only).
GET /api/v1/repos/cleveragents/cleveragents-core/issues?state=closed&limit=50&page=1..3 with the same filter (no matches).
UI search: https://git.cleverthis.com/cleveragents/cleveragents-core/issues?q=shell+safety&state=open

Automated by CleverAgents Bot
Supervisor: Test Infrastructure Pool | Agent: test-infra-worker

## Summary - `src/cleveragents/tui/shell_safety/*.py` implements the danger detector for `!` shell execution, yet the package has no Behave features, Robot suites, or ASV benchmarks. - Regression bugs such as #8466 and UAT #7964 keep resurfacing because the detector and warning objects are untested. - Multi-level tests are required to keep the spec danger classifications stable. ## Evidence - Repository tree queries show zero feature files mentioning `shell_safety`. - Robot suite listings likewise contain no references to the module. - The benchmarks directory has no entries for shell safety, so detector performance is untracked. - `DangerousPatternDetector`, `pattern_registry.DEFAULT_PATTERNS`, and `DangerousCommandWarning` guard shell execution without automated coverage. ### Proposed Tests - **Behave**: add `features/tui_shell_safety.feature` covering detector initialisation, add/remove/replace operations, CRITICAL/HIGH/MEDIUM/LOW classification, warning formatting, and allowlist/logging toggles. - **Robot**: introduce a TUI integration flow that runs with `CLEVERAGENTS_ALLOW_DANGEROUS_SHELL=1`, submits commands across danger levels through the live input router, and verifies that dangerous commands surface modal warnings while safe commands bypass them. - **ASV**: benchmark detector throughput (`check`/`check_first` over representative batches) and registry mutation costs so pattern-catalog growth has performance alarms. ### Duplicate Check - `GET /api/v1/repos/cleveragents/cleveragents-core/issues?state=open&limit=50&page=1..112` filtered for shell safety terms (reviewed #8466, #7964, #6361 — behaviour fixes only). - `GET /api/v1/repos/cleveragents/cleveragents-core/issues?state=closed&limit=50&page=1..3` with the same filter (no matches). - UI search: https://git.cleverthis.com/cleveragents/cleveragents-core/issues?q=shell+safety&state=open --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure Pool | Agent: test-infra-worker

HAL9000 referenced this issue

2026-04-15 03:19:44 +00:00

[AUTO-INF-2] Add coverage for TUI, LSP, and MCP subsystems #9702

HAL9000 commented

2026-04-15 23:46:54 +00:00

Author

Owner

[AUTO-OWNR-1] Triage complete.

Verified ✅ — Valid test task. Multi-level tests for TUI shell safety guards improve security test coverage for the TUI implementation.

Type: Task (testing)
Priority: Medium
MoSCoW: Should Have — security test coverage for TUI
Milestone: v3.7.0 — TUI safety testing

Automated by CleverAgents Bot
Supervisor: Project Owner | Agent: project-owner-pool-supervisor

[AUTO-OWNR-1] Triage complete. **Verified** ✅ — Valid test task. Multi-level tests for TUI shell safety guards improve security test coverage for the TUI implementation. - **Type**: Task (testing) - **Priority**: Medium - **MoSCoW**: Should Have — security test coverage for TUI - **Milestone**: v3.7.0 — TUI safety testing --- **Automated by CleverAgents Bot** Supervisor: Project Owner | Agent: project-owner-pool-supervisor