fix(git_tools): eliminate TOCTOU race in _get_base_env() with double-checked locking
CI / push-validation (pull_request) Successful in 17s
CI / helm (pull_request) Successful in 25s
CI / build (pull_request) Successful in 26s
CI / lint (pull_request) Successful in 26s
CI / quality (pull_request) Successful in 33s
CI / typecheck (pull_request) Successful in 53s
CI / e2e_tests (pull_request) Successful in 4m19s
CI / security (pull_request) Successful in 4m21s
CI / unit_tests (pull_request) Successful in 5m42s
CI / docker (pull_request) Successful in 1m19s
CI / integration_tests (pull_request) Successful in 8m1s
CI / coverage (pull_request) Successful in 11m20s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 57m20s
CI / push-validation (pull_request) Successful in 17s
CI / helm (pull_request) Successful in 25s
CI / build (pull_request) Successful in 26s
CI / lint (pull_request) Successful in 26s
CI / quality (pull_request) Successful in 33s
CI / typecheck (pull_request) Successful in 53s
CI / e2e_tests (pull_request) Successful in 4m19s
CI / security (pull_request) Successful in 4m21s
CI / unit_tests (pull_request) Successful in 5m42s
CI / docker (pull_request) Successful in 1m19s
CI / integration_tests (pull_request) Successful in 8m1s
CI / coverage (pull_request) Successful in 11m20s
CI / status-check (pull_request) Successful in 1s
CI / benchmark-publish (pull_request) Has been skipped
CI / benchmark-regression (pull_request) Successful in 57m20s
Use threading.Lock with double-checked locking to protect the lazy initialisation of the module-level _BASE_ENV cache. Without the lock, two concurrent callers could both observe _BASE_ENV is None, both snapshot os.environ, and write potentially different snapshots — the second write silently discarding the first. The outer `if _BASE_ENV is None` keeps the warm-cache path lock-free; the inner `if` inside `with _BASE_ENV_LOCK` prevents duplicate initialisation on the very first call. Added BDD scenarios in features/git_tools.feature and corresponding step definitions in features/steps/git_tools_steps.py to verify: - Repeated calls return the identical cached dict object - The returned dict contains all expected git override keys - Concurrent initialisation from 20 threads produces exactly one initialisation and all threads receive the same dict object ISSUES CLOSED: #7619
This commit is contained in:
@@ -5,6 +5,18 @@ The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
||||
|
||||
## [Unreleased]
|
||||
|
||||
### Fixed
|
||||
|
||||
- **`git_tools._get_base_env()` TOCTOU race on `_BASE_ENV`** (#7619): Fixed a
|
||||
time-of-check/time-of-use race condition in the lazy initialisation of the
|
||||
module-level `_BASE_ENV` cache. Under concurrent git tool calls (e.g.
|
||||
parallel subplan execution), two threads could both observe `_BASE_ENV is
|
||||
None`, both snapshot `os.environ`, and write potentially different snapshots —
|
||||
the second write silently discarding the first. The fix uses
|
||||
`threading.Lock` with double-checked locking: the outer `if` keeps the
|
||||
warm-cache path lock-free; the inner `if` inside the lock prevents duplicate
|
||||
initialisation on the very first call.
|
||||
|
||||
### Added
|
||||
|
||||
- **TDD Issue-Capture Test Activation** (#7025): Replaced 234 bare `@skip` tags
|
||||
|
||||
@@ -267,3 +267,21 @@ Feature: Built-in Git Tools
|
||||
Given a tool registry
|
||||
When I register all git tools
|
||||
Then all registered git tools should be read-only
|
||||
|
||||
# ---- Thread Safety (_get_base_env TOCTOU fix) ----
|
||||
|
||||
Scenario: _get_base_env returns the same dict object on repeated calls
|
||||
When I call _get_base_env twice in the same thread
|
||||
Then both calls should return the identical dict object
|
||||
|
||||
Scenario: _get_base_env returns a dict containing git override keys
|
||||
When I call _get_base_env
|
||||
Then the returned env dict should contain the key "GIT_PAGER"
|
||||
And the returned env dict should contain the key "NO_COLOR"
|
||||
And the returned env dict should contain the key "GIT_TERMINAL_PROMPT"
|
||||
|
||||
Scenario: _get_base_env is safe under concurrent initialisation
|
||||
Given the base env cache has been reset
|
||||
When I call _get_base_env from 20 concurrent threads
|
||||
Then all threads should receive the same dict object
|
||||
And the base env should have been initialised exactly once
|
||||
|
||||
@@ -6,12 +6,14 @@ import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
from unittest.mock import patch
|
||||
|
||||
from behave import given, then, when
|
||||
|
||||
import cleveragents.tool.builtins.git_tools as _git_tools_module
|
||||
from cleveragents.tool.builtins import register_git_tools
|
||||
from cleveragents.tool.builtins.git_tools import (
|
||||
ALL_GIT_TOOLS,
|
||||
@@ -464,3 +466,98 @@ def step_then_all_read_only(context: Any) -> None:
|
||||
assert tool is not None, f"Tool {spec.name} not found"
|
||||
assert tool.capabilities.read_only is True, f"Tool {spec.name} is not read_only"
|
||||
assert tool.capabilities.writes is False, f"Tool {spec.name} has writes=True"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Thread-safety steps (_get_base_env TOCTOU fix — issue #7619)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@when("I call _get_base_env twice in the same thread")
|
||||
def step_when_call_get_base_env_twice(context: Any) -> None:
|
||||
from cleveragents.tool.builtins.git_tools import _get_base_env
|
||||
|
||||
context.base_env_first = _get_base_env()
|
||||
context.base_env_second = _get_base_env()
|
||||
|
||||
|
||||
@then("both calls should return the identical dict object")
|
||||
def step_then_base_env_same_object(context: Any) -> None:
|
||||
assert context.base_env_first is context.base_env_second, (
|
||||
"_get_base_env() returned different dict objects on repeated calls; "
|
||||
"caching is not working correctly"
|
||||
)
|
||||
|
||||
|
||||
@when("I call _get_base_env")
|
||||
def step_when_call_get_base_env(context: Any) -> None:
|
||||
from cleveragents.tool.builtins.git_tools import _get_base_env
|
||||
|
||||
context.base_env_result = _get_base_env()
|
||||
|
||||
|
||||
@then('the returned env dict should contain the key "{key}"')
|
||||
def step_then_env_dict_contains_key(context: Any, key: str) -> None:
|
||||
assert key in context.base_env_result, (
|
||||
f"Expected key '{key}' in base env dict, but it was absent. "
|
||||
f"Keys present: {list(context.base_env_result.keys())}"
|
||||
)
|
||||
|
||||
|
||||
@given("the base env cache has been reset")
|
||||
def step_given_base_env_cache_reset(context: Any) -> None:
|
||||
# Save the original value so we can restore it after the scenario.
|
||||
context._original_base_env = _git_tools_module._BASE_ENV
|
||||
_git_tools_module._BASE_ENV = None
|
||||
context._cleanup_handlers.append(
|
||||
lambda: setattr(_git_tools_module, "_BASE_ENV", context._original_base_env)
|
||||
)
|
||||
|
||||
|
||||
@when("I call _get_base_env from {count:d} concurrent threads")
|
||||
def step_when_call_get_base_env_concurrent(context: Any, count: int) -> None:
|
||||
from cleveragents.tool.builtins.git_tools import _get_base_env
|
||||
|
||||
results: list[dict[str, str]] = []
|
||||
barrier = threading.Barrier(count)
|
||||
|
||||
def _worker() -> None:
|
||||
# All threads reach the barrier before any calls _get_base_env,
|
||||
# maximising the chance of a concurrent first-call race.
|
||||
barrier.wait()
|
||||
results.append(_get_base_env())
|
||||
|
||||
threads = [threading.Thread(target=_worker) for _ in range(count)]
|
||||
for t in threads:
|
||||
t.start()
|
||||
for t in threads:
|
||||
t.join()
|
||||
|
||||
context.concurrent_base_env_results = results
|
||||
|
||||
|
||||
@then("all threads should receive the same dict object")
|
||||
def step_then_all_threads_same_object(context: Any) -> None:
|
||||
results = context.concurrent_base_env_results
|
||||
assert len(results) > 0, "No results collected from concurrent threads"
|
||||
first = results[0]
|
||||
for i, result in enumerate(results[1:], start=1):
|
||||
assert result is first, (
|
||||
f"Thread {i} received a different dict object than thread 0; "
|
||||
"double-checked locking is not working correctly"
|
||||
)
|
||||
|
||||
|
||||
@then("the base env should have been initialised exactly once")
|
||||
def step_then_base_env_initialised_once(context: Any) -> None:
|
||||
# After all concurrent calls, the module-level _BASE_ENV must be set
|
||||
# (not None) and must be the same object every thread received.
|
||||
assert _git_tools_module._BASE_ENV is not None, (
|
||||
"_BASE_ENV is still None after concurrent _get_base_env() calls"
|
||||
)
|
||||
results = context.concurrent_base_env_results
|
||||
for i, result in enumerate(results):
|
||||
assert result is _git_tools_module._BASE_ENV, (
|
||||
f"Thread {i} result is not the module-level _BASE_ENV; "
|
||||
"multiple initialisations may have occurred"
|
||||
)
|
||||
|
||||
@@ -37,6 +37,7 @@ from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
@@ -63,7 +64,15 @@ _GIT_ENV: dict[str, str] = {
|
||||
}
|
||||
|
||||
# Cached merge of ``os.environ`` with ``_GIT_ENV``, populated on first use.
|
||||
# Protected by ``_BASE_ENV_LOCK`` to prevent a TOCTOU race under concurrent
|
||||
# git tool calls (e.g. parallel subplan execution). Without the lock, two
|
||||
# threads could both observe ``_BASE_ENV is None``, both snapshot
|
||||
# ``os.environ``, and write potentially different snapshots — the second
|
||||
# write silently discarding the first. Double-checked locking ensures the
|
||||
# snapshot is taken exactly once while keeping the fast path (already
|
||||
# initialised) lock-free.
|
||||
_BASE_ENV: dict[str, str] | None = None
|
||||
_BASE_ENV_LOCK: threading.Lock = threading.Lock()
|
||||
|
||||
|
||||
def _get_base_env() -> dict[str, str]:
|
||||
@@ -71,10 +80,17 @@ def _get_base_env() -> dict[str, str]:
|
||||
|
||||
The snapshot is taken on first call and reused thereafter, avoiding a
|
||||
full ``os.environ`` copy on every git invocation.
|
||||
|
||||
Thread-safe via double-checked locking: the outer ``if`` avoids lock
|
||||
acquisition on every call once the cache is warm; the inner ``if``
|
||||
inside the lock prevents duplicate initialisation when two threads race
|
||||
on the very first call.
|
||||
"""
|
||||
global _BASE_ENV
|
||||
if _BASE_ENV is None:
|
||||
_BASE_ENV = {**os.environ, **_GIT_ENV}
|
||||
with _BASE_ENV_LOCK:
|
||||
if _BASE_ENV is None:
|
||||
_BASE_ENV = {**os.environ, **_GIT_ENV}
|
||||
return _BASE_ENV
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
Test Framework: generic
|
||||
Total Tests: 3
|
||||
Passed: 1
|
||||
Failed: 2
|
||||
|
||||
--- Test Results ---
|
||||
✗ Output Block 1
|
||||
✓ nox > Running session typecheck
|
||||
✗ Error Output
|
||||
|
||||
--- Failed Tests ---
|
||||
✗ Output Block 1
|
||||
/tmp/impl-worker-7619-20260413-120000/repo/src/cleveragents/providers/registry.py
|
||||
/tmp/impl-worker-7619-20260413-120000/repo/src/cleveragents/providers/registry.py:529:18 - warning: Import "langchain_groq" could not be resolved from source (reportMissingModuleSource)
|
||||
/tmp/impl-worker-7619-20260413-120000/repo/src/cleveragents/providers/registry.py:537:18 - warning: Import "langchain_together" could not be resolved from source (reportMissingModuleSource)
|
||||
/tmp/impl-worker-7619-20260413-120000/repo/src/cleveragents/providers/registry.py:545:18 - warning: Import "langchain_cohere" could not be resolved from source (reportMissingModuleSource)
|
||||
0 errors, 3 warnings, 0 informations
|
||||
|
||||
✗ Error Output
|
||||
nox > Running session typecheck
|
||||
nox > Creating virtual environment (uv) using python3.13 in .nox/typecheck
|
||||
nox > uv pip install pyright
|
||||
nox > uv pip install -e .
|
||||
nox > pyright
|
||||
nox > Session typecheck was successful in 48 seconds.
|
||||
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
{
|
||||
"framework": "generic",
|
||||
"tests": [
|
||||
{
|
||||
"name": "Output Block 1",
|
||||
"passed": false,
|
||||
"output": [
|
||||
"/tmp/impl-worker-7619-20260413-120000/repo/src/cleveragents/providers/registry.py",
|
||||
" /tmp/impl-worker-7619-20260413-120000/repo/src/cleveragents/providers/registry.py:529:18 - warning: Import \"langchain_groq\" could not be resolved from source (reportMissingModuleSource)",
|
||||
" /tmp/impl-worker-7619-20260413-120000/repo/src/cleveragents/providers/registry.py:537:18 - warning: Import \"langchain_together\" could not be resolved from source (reportMissingModuleSource)",
|
||||
" /tmp/impl-worker-7619-20260413-120000/repo/src/cleveragents/providers/registry.py:545:18 - warning: Import \"langchain_cohere\" could not be resolved from source (reportMissingModuleSource)",
|
||||
"0 errors, 3 warnings, 0 informations"
|
||||
],
|
||||
"rawOutput": "/tmp/impl-worker-7619-20260413-120000/repo/src/cleveragents/providers/registry.py\n /tmp/impl-worker-7619-20260413-120000/repo/src/cleveragents/providers/registry.py:529:18 - warning: Import \"langchain_groq\" could not be resolved from source (reportMissingModuleSource)\n /tmp/impl-worker-7619-20260413-120000/repo/src/cleveragents/providers/registry.py:537:18 - warning: Import \"langchain_together\" could not be resolved from source (reportMissingModuleSource)\n /tmp/impl-worker-7619-20260413-120000/repo/src/cleveragents/providers/registry.py:545:18 - warning: Import \"langchain_cohere\" could not be resolved from source (reportMissingModuleSource)\n0 errors, 3 warnings, 0 informations"
|
||||
},
|
||||
{
|
||||
"name": "nox > Running session typecheck",
|
||||
"passed": true,
|
||||
"output": [
|
||||
"nox > Running session typecheck",
|
||||
"nox > Creating virtual environment (uv) using python3.13 in .nox/typecheck",
|
||||
"nox > uv pip install pyright",
|
||||
"nox > uv pip install -e .",
|
||||
"nox > pyright ",
|
||||
"nox > Session typecheck was successful in 48 seconds."
|
||||
],
|
||||
"rawOutput": "nox > Running session typecheck\nnox > Creating virtual environment (uv) using python3.13 in .nox/typecheck\nnox > uv pip install pyright\nnox > uv pip install -e .\nnox > pyright \nnox > Session typecheck was successful in 48 seconds."
|
||||
},
|
||||
{
|
||||
"name": "Error Output",
|
||||
"passed": false,
|
||||
"output": [
|
||||
"nox > Running session typecheck",
|
||||
"nox > Creating virtual environment (uv) using python3.13 in .nox/typecheck",
|
||||
"nox > uv pip install pyright",
|
||||
"nox > uv pip install -e .",
|
||||
"nox > pyright ",
|
||||
"nox > Session typecheck was successful in 48 seconds.",
|
||||
""
|
||||
],
|
||||
"rawOutput": "nox > Running session typecheck\nnox > Creating virtual environment (uv) using python3.13 in .nox/typecheck\nnox > uv pip install pyright\nnox > uv pip install -e .\nnox > pyright \nnox > Session typecheck was successful in 48 seconds.\n"
|
||||
}
|
||||
],
|
||||
"summary": {
|
||||
"total": 3,
|
||||
"passed": 1,
|
||||
"failed": 2
|
||||
},
|
||||
"rawOutput": "/tmp/impl-worker-7619-20260413-120000/repo/src/cleveragents/providers/registry.py\n /tmp/impl-worker-7619-20260413-120000/repo/src/cleveragents/providers/registry.py:529:18 - warning: Import \"langchain_groq\" could not be resolved from source (reportMissingModuleSource)\n /tmp/impl-worker-7619-20260413-120000/repo/src/cleveragents/providers/registry.py:537:18 - warning: Import \"langchain_together\" could not be resolved from source (reportMissingModuleSource)\n /tmp/impl-worker-7619-20260413-120000/repo/src/cleveragents/providers/registry.py:545:18 - warning: Import \"langchain_cohere\" could not be resolved from source (reportMissingModuleSource)\n0 errors, 3 warnings, 0 informations\n\nnox > Running session typecheck\nnox > Creating virtual environment (uv) using python3.13 in .nox/typecheck\nnox > uv pip install pyright\nnox > uv pip install -e .\nnox > pyright \nnox > Session typecheck was successful in 48 seconds."
|
||||
}
|
||||
Reference in New Issue
Block a user