fix(security): remove eval-based config parsing #318

Closed
opened 2026-02-22 23:41:15 +00:00 by freemo · 1 comment
Owner

Metadata

  • Commit: fix(security): remove eval-based config parsing
  • Branch: feature/m4-security-eval

Subtasks

  • Audit and remove all eval/exec/compile usage from production config paths.
  • Replace any dynamic expression parsing with YAML/JSON parsing and explicit schema validation.
  • Add a hard error if config files contain inline Python or templating directives.
  • Add config scanner that flags disallowed tokens and reports file+line in errors.
  • Add docs/reference/security_eval.md with replacement patterns.
  • Tests (Behave): Add features/security_eval.feature scenarios. (completed by Luis)
  • Tests (Robot): Add robot/security_eval.robot smoke tests.
  • Tests (ASV): Add benchmarks/security_eval_bench.py for config parsing baseline.
  • Verify coverage >=97% via nox -s coverage_report. If coverage is <97% then review the current unit test coverage report at build/coverage.xml and use it to write new Behave based unit tests to improve code coverage. Specifically, write Behave style unit tests that are descriptively named and specifically improves coverage on whichever file has the most uncovered lines by writing tests that will target the uncovered lines in the report. Once that is done rerun nox -s coverage_report to verify all tests pass and coverage is above >=97%. Only mark this as complete once coverage is >=97%, if not repeat this task as many times as is needed until coverage reaches >=97%.
  • Run nox (all default sessions, including benchmark).

Section: ### Section 11: Security & Safety [WORKSTREAM F - Luis + Brent]
Status: Completed

## Metadata - **Commit**: `fix(security): remove eval-based config parsing` - **Branch**: `feature/m4-security-eval` ## Subtasks - [x] Audit and remove all `eval`/`exec`/`compile` usage from production config paths. - [x] Replace any dynamic expression parsing with YAML/JSON parsing and explicit schema validation. - [x] Add a hard error if config files contain inline Python or templating directives. - [x] Add config scanner that flags disallowed tokens and reports file+line in errors. - [x] Add `docs/reference/security_eval.md` with replacement patterns. - [x] Tests (Behave): Add `features/security_eval.feature` scenarios. (completed by Luis) - [x] Tests (Robot): Add `robot/security_eval.robot` smoke tests. - [x] Tests (ASV): Add `benchmarks/security_eval_bench.py` for config parsing baseline. - [x] Verify coverage >=97% via `nox -s coverage_report`. If coverage is <97% then review the current unit test coverage report at `build/coverage.xml` and use it to write new Behave based unit tests to improve code coverage. Specifically, write Behave style unit tests that are descriptively named and specifically improves coverage on whichever file has the most uncovered lines by writing tests that will target the uncovered lines in the report. Once that is done rerun `nox -s coverage_report` to verify all tests pass and coverage is above >=97%. Only mark this as complete once coverage is >=97%, if not repeat this task as many times as is needed until coverage reaches >=97%. - [x] Run `nox` (all default sessions, including benchmark). **Section**: ### Section 11: Security & Safety [WORKSTREAM F - Luis + Brent] **Status**: Completed
freemo added this to the (deleted) milestone 2026-02-22 23:41:15 +00:00
freemo modified the milestone from (deleted) to v3.3.0 2026-02-23 00:07:07 +00:00
Author
Owner

Implementation Notes — Security Hardening: eval() Removal and Config Scanning

2026-02-09: Stage SEC1 Complete - Remove eval() Vulnerability

  • Audit results (7 matches in src/cleveragents/): 2 real vulnerabilities, 4 false positives.
  • Fixes applied: SimpleToolAgent — added named operation registry (_SAFE_OPERATIONS), legacy code blocks fully rejected. ReactiveStreamRouter — added named transform registry, legacy fn string expressions that are not in the registry fully rejected. No eval() or exec() calls remain.

2026-02-10: Stage SEC1.5 Complete - Security BDD tests

  • Created features/security_eval.feature with 8 scenarios covering code injection rejection, malicious transform rejection, valid config acceptance, custom registered operations.

2026-02-19: Stage SEC1.eval COMMIT Complete - Remove eval-based config parsing [Luis]

  • Created config security scanner detecting 15 disallowed patterns across YAML/TOML/generic configs. Reports file path + line number + severity for each violation.
  • Extended security eval feature from 8 to 16 scenarios.

(Migrated from docs/implementation-notes.md)

## Implementation Notes — Security Hardening: eval() Removal and Config Scanning **2026-02-09**: Stage SEC1 Complete - Remove eval() Vulnerability - **Audit results** (7 matches in `src/cleveragents/`): 2 real vulnerabilities, 4 false positives. - **Fixes applied**: `SimpleToolAgent` — added named operation registry (`_SAFE_OPERATIONS`), legacy `code` blocks fully rejected. `ReactiveStreamRouter` — added named transform registry, legacy `fn` string expressions that are not in the registry fully rejected. No `eval()` or `exec()` calls remain. **2026-02-10**: Stage SEC1.5 Complete - Security BDD tests - Created `features/security_eval.feature` with 8 scenarios covering code injection rejection, malicious transform rejection, valid config acceptance, custom registered operations. **2026-02-19**: Stage SEC1.eval COMMIT Complete - Remove eval-based config parsing [Luis] - Created config security scanner detecting 15 disallowed patterns across YAML/TOML/generic configs. Reports file path + line number + severity for each violation. - Extended security eval feature from 8 to 16 scenarios. *(Migrated from `docs/implementation-notes.md`)*
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Blocks
#362 Epic: Security & Safety Hardening
cleveragents/cleveragents-core
Depends on
#137 develop-luis-1
cleveragents/cleveragents-core
Reference
cleveragents/cleveragents-core#318
No description provided.