hurui200320
d878c04745
CI / benchmark-publish (pull_request) Has been skipped
CI / lint (pull_request) Successful in 19s
CI / build (pull_request) Successful in 25s
CI / quality (pull_request) Successful in 30s
CI / typecheck (pull_request) Successful in 49s
CI / security (pull_request) Successful in 1m6s
CI / unit_tests (pull_request) Successful in 3m25s
CI / integration_tests (pull_request) Successful in 4m45s
CI / docker (pull_request) Successful in 1m0s
CI / e2e_tests (pull_request) Successful in 5m24s
CI / coverage (pull_request) Successful in 9m3s
CI / benchmark-regression (pull_request) Has been cancelled
Add robot/e2e/m5_acceptance.robot with 21 zero-mock E2E tests covering
all M5 acceptance criteria: context assembly (add/list/show/clear),
context scaling (10,000+ files without timeout), context policy
configuration (view-specific settings), budget enforcement
(max_file_size/max_total_size constraints), context analysis (ACMS
pipeline inspect/simulate), and plan execution with real LLM calls
(openai/gpt-4o-mini via plan use + plan resume).
Bug fixes discovered during E2E testing:
- Fix _save_policy_json commit bug: session.flush() -> session.commit()
so policy changes persist after session.close() (project_context.py).
Rollback wrapped in contextlib.suppress(Exception) to prevent masking
the original commit exception.
- Add session_factory DI provider to Container as providers.Singleton so
the four project context commands (set/show/inspect/simulate) resolve
a real sessionmaker instead of hitting a missing attribute. Return
type annotated as sessionmaker[Session] using future annotations and
TYPE_CHECKING guard (container.py).
- Propagate GEMINI_API_KEY in nox e2e_tests session (noxfile.py).
- Add Google/Gemini API key pattern (AIzaSy...) to secret redaction
in redaction.py.
Review feedback addressed across eight review passes:
- Budget enforcement test verifies large_file.py exclusion by
max_file_size constraint (not tautological token comparison).
- 10K scaling test validates simulation returns valid fragment_count
(non-negative; non-zero deferred to full ACMS pipeline integration).
- Plan resume test parses JSON via Extract JSON From Stdout with
TRY/EXCEPT — consistent with plan use test pattern.
- __import__("sqlalchemy") replaced with proper import in step defs.
- Context clear test verifies all three files removed (config/main/utils).
- Inspect test asserts tier_metrics and tier_budget are non-empty.
- _save_policy_json rollback wrapped in contextlib.suppress(Exception).
- Skip If No LLM Keys rewritten to use Evaluate with os.environ.get —
keys never stored in any Robot variable.
- All assertion messages follow safe pattern (no stdout/stderr embedding).
- Review finding ID prefixes removed from all comments.
- Type annotations added to all 8 _build_session_factory step functions.
- Default view verification checks exclude-path __pycache__.
- Prerequisite skip guards added to sections 2-4 (Variable Should Exist).
- Plan ID extraction moved inside TRY block.
Results: 25/25 E2E (21 M5 + 2 smoke + 2 M1), 1536/1536 Robot
integration, 383/383 Behave features (10,926 scenarios), coverage 97%.
ISSUES CLOSED: #745
165 lines
9.1 KiB
Gherkin
165 lines
9.1 KiB
Gherkin
Feature: Project context CLI coverage boost
|
|
Additional Behave scenarios targeting uncovered lines in
|
|
src/cleveragents/cli/commands/project_context.py.
|
|
|
|
Background:
|
|
Given a coverage-boost in-memory database is initialized
|
|
And a project "local/cov-app" exists for coverage boost
|
|
|
|
# --- _load_policy_json returns None (line 106) ---
|
|
# --- _read_policy returns default when raw is None (line 145) ---
|
|
# --- _read_acms_config returns defaults when raw is None (line 164) ---
|
|
Scenario: Reading policy from a fresh project returns defaults
|
|
When I read the stored policy for "local/cov-app"
|
|
Then the policy should be a fresh default policy
|
|
And the ACMS config should equal the defaults
|
|
|
|
# --- _write_policy preserves existing ACMS config (lines 199-201) ---
|
|
Scenario: Writing policy without acms_config preserves existing ACMS config
|
|
Given I seed ACMS config for "local/cov-app" with hot_max_tokens 9999
|
|
When I write a policy for "local/cov-app" without explicit ACMS config
|
|
Then the stored project-context ACMS config hot_max_tokens should be 9999
|
|
|
|
# --- clear default view resets to empty ContextView (line 574) ---
|
|
Scenario: Clearing the default view resets it
|
|
When I run coverage-boost context set on "local/cov-app" with view "default" and include-resource "keep-me"
|
|
And I run coverage-boost context set on "local/cov-app" with view "default" and clear flag
|
|
Then the coverage-boost command should succeed
|
|
And the stored coverage-boost default view include_resources should be empty
|
|
|
|
# --- default_depth as non-integer string (lines 609-610) ---
|
|
Scenario: Setting default_depth to a non-integer string keeps it as string
|
|
When I run coverage-boost context set on "local/cov-app" with default_depth "deep"
|
|
Then the coverage-boost command should succeed
|
|
And the stored ACMS config default_depth should be "deep"
|
|
|
|
# --- execution_environment valid (lines 628-648) ---
|
|
Scenario: Setting a valid execution environment persists it
|
|
When I run coverage-boost context set on "local/cov-app" with execution_environment "host"
|
|
Then the coverage-boost command should succeed
|
|
And the stored execution_environment should be "host"
|
|
|
|
# --- execution_environment invalid (lines 632-638) ---
|
|
Scenario: Setting an invalid execution environment fails
|
|
When I run coverage-boost context set on "local/cov-app" with execution_environment "spaceship"
|
|
Then the coverage-boost command should have exit code 1
|
|
And the coverage-boost output should contain "Invalid execution environment"
|
|
|
|
# --- non-rich output format for context_set (line 663) ---
|
|
Scenario: Context set with JSON output format
|
|
When I run coverage-boost context set on "local/cov-app" with view "default" include-resource "res1" and format "json"
|
|
Then the coverage-boost command should succeed
|
|
And the coverage-boost output should be valid JSON
|
|
|
|
# --- invalid view in context_show (lines 704-707) ---
|
|
Scenario: Context show with invalid view fails
|
|
When I run coverage-boost context show on "local/cov-app" with view "bogus"
|
|
Then the coverage-boost command should have exit code 1
|
|
And the coverage-boost output should contain "Invalid view"
|
|
|
|
# --- strategy_filter in context_inspect (lines 874-878) ---
|
|
Scenario: Context inspect with strategy filter narrows fragments
|
|
Given the tier service has fragments for "local/cov-app" with strategy metadata
|
|
When I run coverage-boost context inspect on "local/cov-app" with strategy filter "tier_a"
|
|
Then the coverage-boost command should succeed
|
|
|
|
# --- focus filter in context_inspect (lines 882-885) ---
|
|
Scenario: Context inspect with focus filter narrows fragments
|
|
Given the tier service has fragments for "local/cov-app" with strategy metadata
|
|
When I run coverage-boost context inspect on "local/cov-app" with focus "res:file-alpha"
|
|
Then the coverage-boost command should succeed
|
|
|
|
# --- breadth/depth override in context_inspect (lines 897-902) ---
|
|
Scenario: Context inspect with breadth and depth overrides
|
|
When I run coverage-boost context inspect on "local/cov-app" with breadth 5 and depth "detailed"
|
|
Then the coverage-boost command should succeed
|
|
|
|
# --- inspect with view set appends phase to title (line 937) ---
|
|
Scenario: Context inspect with view set
|
|
When I run coverage-boost context inspect on "local/cov-app" with view "execute"
|
|
Then the coverage-boost command should succeed
|
|
And the coverage-boost output should contain "execute"
|
|
|
|
# --- temporal scope "recent" and "all" in simulate (lines 301, 303) ---
|
|
Scenario: Simulate with temporal scope recent
|
|
Given I seed ACMS config for "local/cov-app" with temporal_scope "recent"
|
|
When I run coverage-boost context simulate on "local/cov-app" with defaults
|
|
Then the coverage-boost command should succeed
|
|
|
|
Scenario: Simulate with temporal scope all
|
|
Given I seed ACMS config for "local/cov-app" with temporal_scope "all"
|
|
When I run coverage-boost context simulate on "local/cov-app" with defaults
|
|
Then the coverage-boost command should succeed
|
|
|
|
# --- budget exceeded breaks loop (line 335), long content truncation (line 339),
|
|
# access_count > 0 relevance (line 344) ---
|
|
Scenario: Simulate truncates fragments exceeding budget and handles long content
|
|
Given the tier service has large fragments for "local/cov-app" exceeding budget
|
|
When I run coverage-boost context simulate on "local/cov-app" with budget 100
|
|
Then the coverage-boost command should succeed
|
|
And the simulation should have fewer fragments than stored
|
|
|
|
# --- simulate with fragments renders rich table (lines 1085-1101) ---
|
|
Scenario: Simulate with fragments shows rich table output
|
|
Given the tier service has fragments for "local/cov-app" with strategy metadata
|
|
When I run coverage-boost context simulate on "local/cov-app" in rich format
|
|
Then the coverage-boost command should succeed
|
|
And the coverage-boost output should contain "Assembled Fragments"
|
|
|
|
# --- invalid temporal_scope in context_set (lines 614-619) ---
|
|
Scenario: Setting an invalid temporal scope fails
|
|
When I run coverage-boost context set on "local/cov-app" with temporal_scope "never"
|
|
Then the coverage-boost command should have exit code 1
|
|
And the coverage-boost output should contain "Invalid temporal scope"
|
|
|
|
# --- context inspect with invalid view (lines 844-848) ---
|
|
Scenario: Context inspect with invalid view fails
|
|
When I run coverage-boost context inspect on "local/cov-app" with view "bogus"
|
|
Then the coverage-boost command should have exit code 1
|
|
And the coverage-boost output should contain "Invalid view"
|
|
|
|
# --- context simulate with invalid view (lines 1032-1036) ---
|
|
Scenario: Context simulate with invalid view fails
|
|
When I run coverage-boost context simulate on "local/cov-app" with view "bogus"
|
|
Then the coverage-boost command should have exit code 1
|
|
And the coverage-boost output should contain "Invalid view"
|
|
|
|
# --- simulate with focus_uris filtering (lines 322-328) ---
|
|
Scenario: Simulate with focus URIs filters fragments
|
|
Given the tier service has fragments for "local/cov-app" with strategy metadata
|
|
When I run coverage-boost context simulate on "local/cov-app" with focus "res:file-alpha"
|
|
Then the coverage-boost command should succeed
|
|
|
|
# --- simulate with strategy_hints (line 293, 361) ---
|
|
Scenario: Simulate with strategy hints
|
|
When I run coverage-boost context simulate on "local/cov-app" with strategy hints "bfs,dfs"
|
|
Then the coverage-boost command should succeed
|
|
|
|
# --- _save_policy_json commit() regression test (line 133) ---
|
|
# Exercises _save_policy_json then reads back via a *real* separate
|
|
# session (no _SafeSession) to prove data persists across session
|
|
# boundaries. If commit() were reverted to flush(), this test fails.
|
|
Scenario: Policy saved with commit persists across separate sessions
|
|
When I save a policy for "local/cov-app" via a real session factory and read it back via a separate session
|
|
Then the policy read from the separate session should contain the saved data
|
|
|
|
# --- context_set ACMS overrides for remaining fields (lines 589-622) ---
|
|
Scenario: Context set with all ACMS pipeline overrides
|
|
When I run coverage-boost context set on "local/cov-app" with all ACMS overrides
|
|
Then the coverage-boost command should succeed
|
|
And the stored ACMS config should reflect all overrides
|
|
|
|
# --- _save_policy_json rollback path (contextlib.suppress + re-raise) ---
|
|
# Verifies that when commit() fails, the exception is re-raised and the
|
|
# session is rolled back, releasing DB locks.
|
|
Scenario: Save policy rollback re-raises after commit failure
|
|
When I attempt to save a policy with a commit that will fail
|
|
Then the original commit exception should be re-raised
|
|
And the session should not be in a dirty state after the failure
|
|
|
|
# --- _save_policy_json on nonexistent project (0 rows affected) ---
|
|
Scenario: Save policy on nonexistent project row updates zero rows
|
|
When I save a policy for a nonexistent project "local/no-such-project"
|
|
Then the save should complete without error
|
|
And the policy should not be retrievable for "local/no-such-project"
|