TDD: LLM agent file_write calls are always blocked — safe_mode: false / context.global.unsafe: true never reach _unsafe_mode #116

New issue

Closed

opened 2026-08-05 18:20:12 +00:00 by CoreRasurae · 0 comments

CoreRasurae commented

2026-08-05 18:20:12 +00:00

Member

Copy link

Metadata

Commit Message: test(agents): capture LLM-agent file_write unsafe-mode regression (#115)
Branch: tdd/m1-llm-tool-unsafe-mode

Background and context

Companion TDD issue for #115 (bug: LLM-agent file_write tool calls are always blocked because cleveractors.agents.llm.LLMAgent's tool-call dispatch reads a config key, unsafe_mode, that nothing in the codebase ever sets — the actor's real safe_mode field, and the actor-level context.global.unsafe → host _unsafe_mode propagation contract, are both ignored). Per the mandatory Bug Fix TDD workflow, this issue must be resolved (failing test merged) before #115's fix is implemented.

Current behavior

No Behave scenario exists that proves file_write is unconditionally blocked for an LLM agent configured with safe_mode: false.

Expected behavior

A Behave scenario exists, tagged @tdd_issue @tdd_issue_115 @tdd_expected_fail, that:

Configures a type: llm agent with tools: [file_write] and safe_mode: false.
Drives (or directly exercises) the tool-call dispatch path in cleveractors.agents.llm.LLMAgent for a file_write call.
Asserts the call succeeds.
The assertion currently fails (bug present) — the scenario passes in CI only because @tdd_expected_fail inverts the result, per the project's TDD tag rules.
The failing assertion uses assert/AssertionError only (never ValueError/RuntimeError/etc.), per the TDD assertion-type rule.

Acceptance criteria

Scenario carries all three tags: @tdd_issue, @tdd_issue_115, @tdd_expected_fail.
Running the scenario's underlying assertion directly (tag inversion aside) fails against current main/master — i.e. it genuinely reproduces #115.
The failing step raises/asserts via AssertionError only.
nox -s unit_tests is green (CI passes via @tdd_expected_fail inversion).
Coverage stays ≥ 97% (nox -s coverage_report).

Supporting information

Bug issue: #115.
Relevant code: cleveractors.agents.llm.LLMAgent._execute_tool_loop (and its two synthesis-retry mirrors), cleveractors.agents.llm.LLMAgent._build_tool_context, cleveractors.agents.tool.ToolAgent._file_write_tool.
Spec: docs/index.md §4.5, §4.5.4, §9.4, §10.3.

Subtasks

Add a Behave scenario (new or extended existing features/*.feature covering LLM tool-calling) exercising file_write via an LLM agent with safe_mode: false.
Implement any missing step definitions in the appropriate features/steps/*_steps.py file (extend an existing shared file if one already covers LLM tool-calling; do not create a new one unnecessarily).
Tag the scenario @tdd_issue @tdd_issue_115 @tdd_expected_fail.
Confirm the scenario fails without tag inversion (bug reproduced) and passes CI with it.
Verify coverage >= 97% via nox -s coverage_report.
Run nox (all default sessions), fix any errors.

Definition of Done

This issue is complete when:

All subtasks above are completed and checked off.
A Git commit is created where the first line matches the Commit Message in Metadata exactly.
The commit is pushed to the branch matching the Branch in Metadata exactly.
The commit is submitted as a PR to master, reviewed, and merged.
CI passes with the scenario's @tdd_expected_fail inversion in effect.

## Metadata - **Commit Message:** test(agents): capture LLM-agent file_write unsafe-mode regression (#115) - **Branch:** tdd/m1-llm-tool-unsafe-mode ## Background and context Companion TDD issue for #115 (bug: LLM-agent `file_write` tool calls are always blocked because `cleveractors.agents.llm.LLMAgent`'s tool-call dispatch reads a config key, `unsafe_mode`, that nothing in the codebase ever sets — the actor's real `safe_mode` field, and the actor-level `context.global.unsafe` → host `_unsafe_mode` propagation contract, are both ignored). Per the mandatory Bug Fix TDD workflow, this issue must be resolved (failing test merged) before #115's fix is implemented. ## Current behavior No Behave scenario exists that proves `file_write` is unconditionally blocked for an LLM agent configured with `safe_mode: false`. ## Expected behavior A Behave scenario exists, tagged `@tdd_issue @tdd_issue_115 @tdd_expected_fail`, that: - Configures a `type: llm` agent with `tools: [file_write]` and `safe_mode: false`. - Drives (or directly exercises) the tool-call dispatch path in `cleveractors.agents.llm.LLMAgent` for a `file_write` call. - Asserts the call succeeds. - The assertion currently **fails** (bug present) — the scenario passes in CI only because `@tdd_expected_fail` inverts the result, per the project's TDD tag rules. - The failing assertion uses `assert`/`AssertionError` only (never `ValueError`/`RuntimeError`/etc.), per the TDD assertion-type rule. ## Acceptance criteria - [ ] Scenario carries all three tags: `@tdd_issue`, `@tdd_issue_115`, `@tdd_expected_fail`. - [ ] Running the scenario's underlying assertion directly (tag inversion aside) fails against current `main`/`master` — i.e. it genuinely reproduces #115. - [ ] The failing step raises/asserts via `AssertionError` only. - [ ] `nox -s unit_tests` is green (CI passes via `@tdd_expected_fail` inversion). - [ ] Coverage stays ≥ 97% (`nox -s coverage_report`). ## Supporting information - Bug issue: #115. - Relevant code: `cleveractors.agents.llm.LLMAgent._execute_tool_loop` (and its two synthesis-retry mirrors), `cleveractors.agents.llm.LLMAgent._build_tool_context`, `cleveractors.agents.tool.ToolAgent._file_write_tool`. - Spec: `docs/index.md` §4.5, §4.5.4, §9.4, §10.3. ## Subtasks - [ ] Add a Behave scenario (new or extended existing `features/*.feature` covering LLM tool-calling) exercising `file_write` via an LLM agent with `safe_mode: false`. - [ ] Implement any missing step definitions in the appropriate `features/steps/*_steps.py` file (extend an existing shared file if one already covers LLM tool-calling; do not create a new one unnecessarily). - [ ] Tag the scenario `@tdd_issue @tdd_issue_115 @tdd_expected_fail`. - [ ] Confirm the scenario fails without tag inversion (bug reproduced) and passes CI with it. - [ ] Verify coverage >= 97% via `nox -s coverage_report`. - [ ] Run `nox` (all default sessions), fix any errors. ## Definition of Done This issue is complete when: - All subtasks above are completed and checked off. - A Git commit is created where the first line matches the Commit Message in Metadata exactly. - The commit is pushed to the branch matching the Branch in Metadata exactly. - The commit is submitted as a PR to master, reviewed, and merged. - CI passes with the scenario's `@tdd_expected_fail` inversion in effect.

CoreRasurae added the

labels

2026-08-05 18:20:12 +00:00

CoreRasurae added a new dependency

2026-08-05 18:21:17 +00:00

#115 LLM agent file_write calls are always blocked — safe_mode: false / context.global.unsafe: true never reach _unsafe_mode