brent.edwards
eb46f0ff54
CI / push-validation (push) Successful in 40s
CI / helm (push) Successful in 44s
CI / build (push) Successful in 1m14s
CI / lint (push) Successful in 1m18s
CI / quality (push) Successful in 1m36s
CI / e2e_tests (push) Successful in 1m19s
CI / typecheck (push) Successful in 2m22s
CI / security (push) Successful in 2m23s
CI / benchmark-regression (push) Failing after 41s
CI / integration_tests (push) Successful in 4m46s
CI / unit_tests (push) Failing after 20m13s
CI / benchmark-publish (push) Failing after 28m28s
CI / coverage (push) Has been cancelled
CI / docker (push) Has been cancelled
CI / status-check (push) Has been cancelled
## Summary This PR fixes issue #10878 where architecture reviews were truncated because the regex pattern for parsing file output would stop at the first ``` encountered in the Markdown report. ## Changes - Change file delimiters from ``` to >>>>>>>/<<<<<<< to avoid Markdown conflicts - Add tier hydration before strategize phase in plan_executor.py - Increase max_tokens to 16384 in llm_actors.py for longer outputs - Increase context_max_tokens_hot from 16000 to 32000 in settings.py - Fix get_hot_view → get_hot_fragments in strategy_actor.py and plan_executor.py - Add opencode to skip directories in context_tier_hydrator.py - Change sandbox output location to plan-output/ directory in plan.py - Add get_context_summary stub method to acms_service.py ## Testing Run architecture review action and verify the output report is complete with all sections. Reviewed-on: #10938
87 lines
4.7 KiB
Gherkin
87 lines
4.7 KiB
Gherkin
@tdd @tdd_issue @tdd_issue_10878
|
|
Feature: Delimiter _parse_file_blocks handles embedded markdown code fences (#10878)
|
|
Bug #10878 caused architecture reviews to be truncated because the old
|
|
backtick delimiter pattern stopped at the first ``` appearing *inside*
|
|
LLM-generated file content. All subsequent sections were lost - for
|
|
example a report with 9 TOC entries produced only 3 before truncation.
|
|
|
|
The fix introduces unique sentinel markers:
|
|
<<<<<<< CLEVERAGENTS_FILE_START >>>>>>> / <<<<<<< CLEVERAGENTS_FILE_END >>>>>>>
|
|
|
|
This regression test proves the new sentinel delimiters do NOT collide
|
|
when file bodies contain triple-backtick markdown code fences.
|
|
|
|
Forgejo: #10878
|
|
|
|
|
|
# -------------------------------------------------------------------
|
|
# Main regression - embedded ```python code fences inside FILE blocks
|
|
# -------------------------------------------------------------------
|
|
|
|
@tdd_issue @tdd_issue_10878
|
|
Scenario: New delimiter parses all FILE blocks when each body contains embedded ```python code fences
|
|
Given an LLM response with two FILE blocks, each having ```python sections in the body
|
|
And a mock plan_id "01HQREGTEST"
|
|
When I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern
|
|
Then it should return 2 changeset entries
|
|
|
|
|
|
# -------------------------------------------------------------------
|
|
# Real-world architecture-review pattern (the original bug)
|
|
# -------------------------------------------------------------------
|
|
|
|
@tdd_issue @tdd_issue_10878
|
|
Scenario: Full regression - report header + three FILE blocks each with inline markdown code fences
|
|
Given an LLM response mimicking an architecture review with report prose and three FILE blocks each containing markdown code example triple-backticks in their body
|
|
And a mock plan_id "01HQARCHTEST"
|
|
When I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern
|
|
Then it should return the full entry count of all embedded blocks
|
|
|
|
|
|
# -------------------------------------------------------------------
|
|
# Edge case - trailing ``` at end of file body before END marker
|
|
# -------------------------------------------------------------------
|
|
|
|
@tdd_issue @tdd_issue_10878
|
|
Scenario: A FILE block whose last line before the END sentinel is exactly ``` parses correctly
|
|
Given an LLM response where a single FILE block body ends on a line that is exactly ``` after code content
|
|
And a mock plan_id "01HQTRAILT"
|
|
When I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern
|
|
Then it should return 1 changeset entries
|
|
|
|
|
|
# -------------------------------------------------------------------
|
|
# Edge case - sentinel text mentioned inline without full markers
|
|
# -------------------------------------------------------------------
|
|
|
|
@tdd_issue @tdd_issue_10878
|
|
Scenario: LLM body mentions 'CLEVERAGENTS_FILE_END' in prose but does not use the full sentinel marker
|
|
Given an LLM response where a FILE block body contains the word 'CLEVERAGENTS_FILE_END' referenced in prose within content and another file follows
|
|
And a mock plan_id "01HQSENTINEL"
|
|
When I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern
|
|
Then it should return 2 changeset entries (second entry has content in body)
|
|
|
|
|
|
# -------------------------------------------------------------------
|
|
# Edge case - git merge-conflict markers alongside new delimiters
|
|
# -------------------------------------------------------------------
|
|
|
|
@tdd_issue @tdd_issue_10878
|
|
Scenario: FILE block bodies contain git merge-conflict-style markers but only the CLEVERAGENTS sentinels match
|
|
Given an LLM response where FILE block bodies include text like '>>>>>>> some_branch' and '<<<<<<< base' but the actual delimiters are CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END markers
|
|
And a mock plan_id "01HQGITTEST"
|
|
When I parse file blocks using the new CLEVERAGENTS_FILE_START / CLEVERAGENTS_FILE_END delimiter pattern
|
|
Then it should return 2 changeset entries
|
|
|
|
|
|
# -------------------------------------------------------------------
|
|
# Escape support for delimiter markers in file content (#262743)
|
|
# -------------------------------------------------------------------
|
|
|
|
@tdd_issue @tdd_issue_10878
|
|
Scenario: Escaped marker occurrences inside file body are not treated as boundaries
|
|
Given an LLM response with a single FILE block using legacy markers where the body contains a backslash-escaped <<<<<<< CLEVERAGENTS_FILE_START >>>>>>> sequence that should be preserved as literal text
|
|
And a mock plan_id "01HQESCTEST"
|
|
When I parse file blocks using the new delimiter pattern
|
|
Then it should return 1 changeset entries
|