perf(test): replace per-instance _database_initialized flag with process-global cache #1264
No reviewers
Labels
No labels
auto/needs-reevaluation
controller-managed
auto/blocked-by-deps
auto/ci-timeout
auto/claimed-implementer
auto/claimed-merge
auto/claimed-reviewer
auto/driver-down
auto/invariant-violation
auto/last-attempt-tier-0
auto/last-attempt-tier-1
auto/last-attempt-tier-2
auto/last-attempt-tier-min
Automation Tracking
auto/needs-conflict-resolution
auto/needs-implementer
auto/postmortem
auto/ready-to-merge
auto/restart-throttled
auto/revert
auto/sentinel
auto/stale-inactivity
auto/unstable
Blocked
Bounty
$100
Bounty
$1000
Bounty
$10000
Bounty
$20
Bounty
$2000
Bounty
$250
Bounty
$50
Bounty
$500
Bounty
$5000
Bounty
$750
MoSCoW
Could have
MoSCoW
Must have
MoSCoW
Should have
Needs Feedback
Points
1
Points
13
Points
2
Points
21
Points
3
Points
34
Points
5
Points
55
Points
8
Points
88
Priority
Backlog
Priority
CI Blocker
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Signed-off: Owner
Signed-off: Scrum Master
Signed-off: Tech Lead
Spike
State
Completed
State
Duplicate
State
In Progress
State
In Review
State
Paused
State
Unverified
State
Verified
State
Wont Do
Type
Automation
Type
Bug
Type
Discussion
Type
Documentation
Type
Epic
Type
Feature
Type
Legendary
Type
Refactor
Type
Support
Type
Task
Type
Testing
No project
No assignees
2 participants
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
cleveragents/cleveragents-core!1264
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "perf/m3-global-db-initialized-cache"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
Adds a process-global
_INITIALIZED_DBS: set[str]cache tofeatures/environment.pythat short-circuits repeated_fast_init_or_upgradecalls for the same database URL. This eliminates ~65,000 unnecessary function-body executions per full test run (URL parsing, path extraction, prefix matching, andstat()syscalls) — approximately 7UnitOfWorkinstantiations × 10,700 scenarios.Changes
features/environment.py_INITIALIZED_DBS: set[str] = set()module-level set with documentation referencing issue #735._fast_init_or_upgradeclosure:if db_url in _INITIALIZED_DBS: returnearly check at the top (before any URL parsing)._INITIALIZED_DBS.add(db_url)after both successful paths (non-empty file detection and template copy)._INITIALIZED_DBS.clear()inbefore_scenarioto prevent cross-scenario state leaks.features/fast_init_upgrade.feature@mock_onlyscenarios for the global cache:features/steps/fast_init_upgrade_steps.py_get_initialized_dbs()helper that accesses the cache through_fast_init_or_upgrade.__globals__to correctly resolve Behave'sexec_file()namespace isolation.Design Decisions
setover thread-local: Behave runs scenarios sequentially within a worker process; thread safety is not a concern._original_init_or_upgradeand are not cached.before_scenario: Each scenario gets fresh temp-DB paths, so prior entries are stale and would waste memory.__globals__access in tests: Behave loadsenvironment.pyviaexec_file()into a private hooks dict, creating a separate namespace fromfeatures.environment.__dict__. Test steps read the cache through the closure function's__globals__to access the same set object.Quality Gates
nox -s lintnox -s typechecknox -s unit_tests(parallel)nox -s unit_tests(serial)nox -s integration_testsnox -s coverage_reportnox -s benchmarknox -s security_scannox -s dead_codenox -s docsnox -s buildCloses #735
🔒 Claimed by pr-reviewer-1. Starting independent code review.
Code Review — APPROVED ✅
Summary
Clean, well-implemented performance optimization that adds a process-global
_INITIALIZED_DBS: set[str]cache tofeatures/environment.py, eliminating ~65,000 unnecessary_fast_init_or_upgradefunction-body executions per full test run.Review Findings
Specification Alignment ✅
Correctness ✅
_fast_init_or_upgrade, before any URL parsing — maximizes performance benefit.startswith("sqlite")check) and are never cached.before_scenario— correct, since each scenario gets fresh temp-DB paths.Thread Safety ✅
setis appropriate: Behave runs scenarios sequentially within a worker process. In parallel mode (behave-parallel), each worker is a separate process with its own_INITIALIZED_DBSset.Test Quality ✅
@mock_onlyBehave scenarios cover: cache hit for non-empty DB, cache hit after template copy, cache clearing between scenarios, and non-caching of non-matching-prefix URLs._get_initialized_dbs()helper correctly uses__globals__to access the cache through the closure's namespace (necessary due to Behave'sexec_file()loading), with a clear docstring explaining why.Code Quality ✅
Commit Message ✅
ISSUES CLOSED: #735footer.Security ✅
No issues found. Approving for merge.
Review claimed by reviewer pool instance reviewer-pool-1. Dispatching independent code review.
Independent Code Review — APPROVED ✅
Reviewer: reviewer-pool-1 (second independent review)
Summary
Clean, well-scoped performance optimization that adds a process-global
_INITIALIZED_DBS: set[str]cache tofeatures/environment.py, eliminating ~65,000 redundant_fast_init_or_upgradefunction-body executions per full test run. No production code is touched — this is purely test infrastructure.Review Findings
Specification Alignment ✅
Correctness ✅
_fast_init_or_upgrade, before any URL parsing — maximizes the performance benefit.sqlite://, and non-matching-prefix URLs correctly delegate to the original and are never cached — these are the right exclusions.startswith("sqlite")check and are never cached.before_scenario— correct placement since each scenario gets fresh temp-DB paths. If TDD tag validation fails andbefore_scenarioreturns early before the clear, the next scenario'sbefore_scenariowill clear it, so no cross-scenario leak is possible.Thread Safety ✅
setis appropriate: Behave runs scenarios sequentially within a worker process. In parallel mode (behave-parallel), each worker is a separate process with its own_INITIALIZED_DBSset — no shared state.Test Quality ✅
@mock_onlyBehave scenarios cover all important cache behaviors:_get_initialized_dbs()helper correctly uses__globals__to access the cache through the closure's namespace (necessary due to Behave'sexec_file()loading), with a clear docstring explaining the rationale.Type Safety ✅
_INITIALIZED_DBS: set[str] = set()— properly annotated module-level variable.# type: ignoresuppressions added (the existing one on L510 is pre-existing for the monkey-patch assignment).Commit Message ✅
perf(test): replace per-instance _database_initialized flag with process-global cacheISSUES CLOSED: #735footer.PR Metadata ✅
Type/Tasklabel present.Closes #735in PR body.Security ✅
Code Quality ✅
No issues found. Approving for merge.