TDD: Inline-code sandbox exposes import, letting code bypass the Restricted Built-ins set entirely (e.g. import os) #108

New issue

Closed

opened 2026-08-04 18:12:43 +00:00 by CoreRasurae · 0 comments

CoreRasurae commented

2026-08-04 18:12:43 +00:00

Member

Metadata

Commit Message: test(agents): capture inline sandbox __import__ escape regression (#107)
Branch: tdd/m1-inline-sandbox-import-restriction

Background and context

Companion TDD issue for #107 ("Inline-code sandbox exposes __import__, letting code bypass the Restricted Built-ins set entirely"). Per this project's mandatory bug-fix TDD workflow, this issue's job is to add a failing (currently bug-present) Behave scenario proving the sandbox escape exists, tagged @tdd_issue @tdd_issue_107 @tdd_expected_fail, before any fix is written on a bugfix/ branch.

Current behavior

ToolAgent._execute_python_code's __builtins__ includes a working __import__, so an inline-code body can execute import os (or any other module) and use it freely. See #107 for the full technical analysis and spec references.

Expected behavior

A Behave scenario exists that executes inline code containing import os and asserts that this MUST fail with a restricted-import error. While the bug is present, that assertion itself fails (since import os currently succeeds) — the @tdd_expected_fail tag inverts this so CI passes on the proof of the bug, not on the underlying (currently insecure) behavior. The failing step uses assert/AssertionError only — never ImportError, ValueError, or any other non-assertion exception — so the TDD CI hook inverts it correctly.

Acceptance criteria

The scenario carries all three required tags: @tdd_issue, @tdd_issue_107, @tdd_expected_fail.
The scenario's failing step raises/asserts via AssertionError only.
Running the scenario with @tdd_expected_fail removed locally demonstrably fails today (proving the bug is real), confirmed before merge.
The scenario is merged to master via its own PR before work begins on #107's fix.

Supporting information

See #107 for the full technical analysis and spec references (docs/index.md §13.2.1, §13.2.3).
Forgejo dependency: #107 depends on this issue (this issue blocks #107), per the project's TDD-before-fix workflow.

Subtasks

Write a Behave scenario (e.g. in features/tool_agent.feature or a dedicated file) that executes inline code containing import os, tagged @tdd_issue @tdd_issue_107 @tdd_expected_fail.
Confirm the scenario fails (proving the bug) when @tdd_expected_fail is temporarily removed locally.
Run nox (all default sessions) to confirm the scenario integrates cleanly (passes via the tdd-expected-fail inversion).
No CHANGELOG entry — test-only issues do not get CHANGELOG entries per project convention.

Definition of Done

This issue is complete when:

The tagged scenario is committed with the exact prescribed commit message first line.
The commit is pushed to the branch matching the Branch in Metadata exactly.
The commit is submitted as a PR to master, reviewed, and merged.
This TDD issue is closed, with #107 depending on it (Forgejo dependency set at creation).

## Metadata - **Commit Message:** `test(agents): capture inline sandbox __import__ escape regression (#107)` - **Branch:** `tdd/m1-inline-sandbox-import-restriction` ## Background and context Companion TDD issue for #107 ("Inline-code sandbox exposes `__import__`, letting code bypass the Restricted Built-ins set entirely"). Per this project's mandatory bug-fix TDD workflow, this issue's job is to add a failing (currently bug-present) Behave scenario proving the sandbox escape exists, tagged `@tdd_issue @tdd_issue_107 @tdd_expected_fail`, before any fix is written on a `bugfix/` branch. ## Current behavior `ToolAgent._execute_python_code`'s `__builtins__` includes a working `__import__`, so an inline-code body can execute `import os` (or any other module) and use it freely. See #107 for the full technical analysis and spec references. ## Expected behavior A Behave scenario exists that executes inline code containing `import os` and asserts that this MUST fail with a restricted-import error. While the bug is present, that assertion itself fails (since `import os` currently succeeds) — the `@tdd_expected_fail` tag inverts this so CI passes on the *proof of the bug*, not on the underlying (currently insecure) behavior. The failing step uses `assert`/`AssertionError` only — never `ImportError`, `ValueError`, or any other non-assertion exception — so the TDD CI hook inverts it correctly. ## Acceptance criteria - [ ] The scenario carries all three required tags: `@tdd_issue`, `@tdd_issue_107`, `@tdd_expected_fail`. - [ ] The scenario's failing step raises/asserts via `AssertionError` only. - [ ] Running the scenario with `@tdd_expected_fail` removed locally demonstrably fails today (proving the bug is real), confirmed before merge. - [ ] The scenario is merged to `master` via its own PR before work begins on #107's fix. ## Supporting information - See #107 for the full technical analysis and spec references (`docs/index.md` §13.2.1, §13.2.3). - Forgejo dependency: #107 depends on this issue (this issue blocks #107), per the project's TDD-before-fix workflow. ## Subtasks - [ ] Write a Behave scenario (e.g. in `features/tool_agent.feature` or a dedicated file) that executes inline code containing `import os`, tagged `@tdd_issue @tdd_issue_107 @tdd_expected_fail`. - [ ] Confirm the scenario fails (proving the bug) when `@tdd_expected_fail` is temporarily removed locally. - [ ] Run `nox` (all default sessions) to confirm the scenario integrates cleanly (passes via the tdd-expected-fail inversion). - [ ] No CHANGELOG entry — test-only issues do not get CHANGELOG entries per project convention. ## Definition of Done This issue is complete when: - The tagged scenario is committed with the exact prescribed commit message first line. - The commit is pushed to the branch matching the Branch in Metadata exactly. - The commit is submitted as a PR to `master`, reviewed, and merged. - This TDD issue is closed, with #107 depending on it (Forgejo dependency set at creation).

CoreRasurae added the

Rows
Columns

TDD: Inline-code sandbox exposes __import__, letting code bypass the Restricted Built-ins set entirely (e.g. import os) #108