TEST-INFRA: [test-data-quality] `context.temp_dir` created in step files is never cleaned up by `after_scenario` #10082

New issue

Open

opened 2026-04-16 22:13:20 +00:00 by HAL9000 · 0 comments

HAL9000 commented

2026-04-16 22:13:20 +00:00

Owner

Metadata

Branch: test/test-data-quality-temp-dir-cleanup
Commit Message: test(infra): add context.temp_dir cleanup to after_scenario hook to prevent test isolation leaks
Milestone: v3.5.0
Parent Epic: —

Background and Context

The after_scenario hook in features/environment.py cleans up context.test_dir when it exists, but does NOT clean up context.temp_dir. However, 80+ step files create temporary directories via tempfile.mkdtemp() and store them in context.temp_dir. These directories are never removed after each scenario, causing:

Test isolation failures: Leftover directories from one scenario can interfere with subsequent scenarios if they share path patterns or if the OS reuses temp directory names.
Disk space leaks: Each test run accumulates hundreds of uncleaned temporary directories in the system temp directory.
Parallel test pollution: In parallel test runs (behave-parallel), leaked temp directories from one worker can be accessed by another.

Evidence: In features/environment.py, after_scenario only handles context.test_dir:

if hasattr(context, "test_dir") and context.test_dir:
    try:
        if Path(context.test_dir).exists():
            shutil.rmtree(context.test_dir)
    except Exception:
        pass
    context.test_dir = None

But context.temp_dir is created in 80+ step files (e.g., plan_service_steps.py, project_service_steps.py, actor_service_steps.py) with 184+ usages and 338+ mkdtemp/mktemp calls, none of which are cleaned up by after_scenario.

Example from plan_service_steps.py:

context.temp_dir = Path(tempfile.mkdtemp())
# ... used throughout the scenario ...
# Never cleaned up in after_scenario!

Expected Behavior

after_scenario in features/environment.py should clean up context.temp_dir (mirroring the existing context.test_dir cleanup pattern) so that no temporary directories from context.temp_dir are left on disk after test scenarios complete.

Acceptance Criteria

after_scenario in features/environment.py cleans up context.temp_dir (and associated WAL/journal files) when it exists, mirroring the context.test_dir cleanup pattern
No temporary directories from context.temp_dir are left on disk after test scenarios complete
No step file manually cleans up context.temp_dir in a way that conflicts with the new hook cleanup
All nox stages pass
Coverage >= 97%

Subtasks

Add context.temp_dir cleanup to after_scenario in features/environment.py (mirror the existing context.test_dir cleanup pattern)
Audit all step files that set context.temp_dir to confirm they rely on after_scenario for cleanup (not inline cleanup)
Verify no step file manually cleans up context.temp_dir in a way that would conflict with the new hook cleanup
Run nox -s unit_tests to confirm no regressions
Run nox -s coverage_report to confirm coverage >= 97%

Definition of Done

after_scenario in features/environment.py cleans up context.temp_dir (and associated WAL/journal files) when it exists
No temporary directories from context.temp_dir are left on disk after test scenarios complete
All nox stages pass
Coverage >= 97%

Duplicate Check

Searched open issues for: "temp_dir", "fixture cleanup", "after_scenario", "test isolation", "context.temp_dir cleanup"

Issue #5844: "Replace hardcoded /tmp paths in test fixtures" — covers hardcoded paths in fixture JSON files, NOT the context.temp_dir cleanup gap in after_scenario. Different issue.
Issue #3063: "Replace hardcoded temporary paths in test fixtures" — same as #5844, different issue.
Issue #7087: "Critical race condition using deprecated tempfile.mktemp()" — covers mktemp() race condition in environment.py, NOT the missing context.temp_dir cleanup. Different issue.
Issue #729: "remove dead after_scenario hooks from step files" — covers dead hooks in step files, NOT the missing context.temp_dir cleanup in environment.py. Different issue.
Issue #8420: "cli_plan_context_commands_steps.py defines after_scenario hook outside environment.py" — covers misplaced hooks, NOT the missing context.temp_dir cleanup. Different issue.

No existing issue tracks the specific gap that after_scenario cleans context.test_dir but not context.temp_dir.

Automated by CleverAgents Bot
Supervisor: Test Infrastructure Pool | Agent: test-infra-pool-supervisor

## Metadata - **Branch**: `test/test-data-quality-temp-dir-cleanup` - **Commit Message**: `test(infra): add context.temp_dir cleanup to after_scenario hook to prevent test isolation leaks` - **Milestone**: v3.5.0 - **Parent Epic**: — ## Background and Context The `after_scenario` hook in `features/environment.py` cleans up `context.test_dir` when it exists, but does NOT clean up `context.temp_dir`. However, 80+ step files create temporary directories via `tempfile.mkdtemp()` and store them in `context.temp_dir`. These directories are never removed after each scenario, causing: 1. **Test isolation failures**: Leftover directories from one scenario can interfere with subsequent scenarios if they share path patterns or if the OS reuses temp directory names. 2. **Disk space leaks**: Each test run accumulates hundreds of uncleaned temporary directories in the system temp directory. 3. **Parallel test pollution**: In parallel test runs (behave-parallel), leaked temp directories from one worker can be accessed by another. **Evidence**: In `features/environment.py`, `after_scenario` only handles `context.test_dir`: ```python if hasattr(context, "test_dir") and context.test_dir: try: if Path(context.test_dir).exists(): shutil.rmtree(context.test_dir) except Exception: pass context.test_dir = None ``` But `context.temp_dir` is created in 80+ step files (e.g., `plan_service_steps.py`, `project_service_steps.py`, `actor_service_steps.py`) with 184+ usages and 338+ `mkdtemp`/`mktemp` calls, none of which are cleaned up by `after_scenario`. Example from `plan_service_steps.py`: ```python context.temp_dir = Path(tempfile.mkdtemp()) # ... used throughout the scenario ... # Never cleaned up in after_scenario! ``` ## Expected Behavior `after_scenario` in `features/environment.py` should clean up `context.temp_dir` (mirroring the existing `context.test_dir` cleanup pattern) so that no temporary directories from `context.temp_dir` are left on disk after test scenarios complete. ## Acceptance Criteria - [ ] `after_scenario` in `features/environment.py` cleans up `context.temp_dir` (and associated WAL/journal files) when it exists, mirroring the `context.test_dir` cleanup pattern - [ ] No temporary directories from `context.temp_dir` are left on disk after test scenarios complete - [ ] No step file manually cleans up `context.temp_dir` in a way that conflicts with the new hook cleanup - [ ] All nox stages pass - [ ] Coverage >= 97% ## Subtasks - [ ] Add `context.temp_dir` cleanup to `after_scenario` in `features/environment.py` (mirror the existing `context.test_dir` cleanup pattern) - [ ] Audit all step files that set `context.temp_dir` to confirm they rely on `after_scenario` for cleanup (not inline cleanup) - [ ] Verify no step file manually cleans up `context.temp_dir` in a way that would conflict with the new hook cleanup - [ ] Run `nox -s unit_tests` to confirm no regressions - [ ] Run `nox -s coverage_report` to confirm coverage >= 97% ## Definition of Done - [ ] `after_scenario` in `features/environment.py` cleans up `context.temp_dir` (and associated WAL/journal files) when it exists - [ ] No temporary directories from `context.temp_dir` are left on disk after test scenarios complete - [ ] All nox stages pass - [ ] Coverage >= 97% ## Duplicate Check Searched open issues for: "temp_dir", "fixture cleanup", "after_scenario", "test isolation", "context.temp_dir cleanup" - Issue #5844: "Replace hardcoded /tmp paths in test fixtures" — covers hardcoded paths in fixture JSON files, NOT the `context.temp_dir` cleanup gap in `after_scenario`. Different issue. - Issue #3063: "Replace hardcoded temporary paths in test fixtures" — same as #5844, different issue. - Issue #7087: "Critical race condition using deprecated tempfile.mktemp()" — covers `mktemp()` race condition in `environment.py`, NOT the missing `context.temp_dir` cleanup. Different issue. - Issue #729: "remove dead after_scenario hooks from step files" — covers dead hooks in step files, NOT the missing `context.temp_dir` cleanup in `environment.py`. Different issue. - Issue #8420: "cli_plan_context_commands_steps.py defines after_scenario hook outside environment.py" — covers misplaced hooks, NOT the missing `context.temp_dir` cleanup. Different issue. No existing issue tracks the specific gap that `after_scenario` cleans `context.test_dir` but not `context.temp_dir`. --- **Automated by CleverAgents Bot** Supervisor: Test Infrastructure Pool | Agent: test-infra-pool-supervisor

HAL9000 added the

labels

2026-04-16 22:20:27 +00:00

HAL9000 referenced this issue

2026-04-17 10:30:45 +00:00