4 Commits

Author	SHA1	Message	Date
CoreRasurae	3e13411fcf	fix(actors): distinguish namespace/name from provider/model in actor name parsing ... When action YAML references actors using namespace/name format (e.g. `strategy_actor: local/my-strategist`), `_parse_actor_name()` incorrectly treated the namespace prefix as a provider name, causing `ValueError: Unknown provider type: local`. Changes: - `_is_known_provider()`: New utility function (strategy_resolution.py) that checks whether a slash-separated first segment matches a known `ProviderType` value (openai, anthropic, etc.). Consolidated into a single implementation; llm_actors.py now imports from strategy_resolution.py. - `_parse_actor_name()`: When the first segment IS a known provider, preserves existing behaviour (provider/model). When it is NOT a known provider, treats the input as namespace/name and returns the full name as the model identifier with the default provider. Callers SHOULD pre-resolve namespace/name references via the actor registry before calling this function. Both implementations (strategy_resolution.py and llm_actors.py) updated; ProviderType import moved to top level. - `PlanLifecycleService.resolve_actor_provider_model()`: New method that resolves a namespaced actor name (e.g. local/my-strategist) to provider/model format by looking up the actor record and extracting its provider and model fields. - `LifecycleService` Protocol (strategy_resolution.py): Added `resolve_actor_provider_model` to the protocol, eliminating the need for `# type: ignore[union-attr]` at call sites. - `PlanLifecycleProtocol` (llm_actors.py): Added `resolve_actor_provider_model` to the protocol. - Caller pre-resolution: StrategyActor, LLMStrategizeActor, LLMExecuteActor, SessionWorkflow._resolve_llm(), and CLI session commands now pre-resolve actor names through the lifecycle service or via injected actor_resolver callables before calling `_parse_actor_name()`, ensuring namespace/name references are correctly mapped to their underlying LLM providers. - `_build_actor_resolver()` (cli/commands/session.py) and `_build_actor_resolver_for_session_workflow()` (a2a/facade.py): Moved `_is_known_provider` imports to top level; removed redundant `except (NotFoundError, Exception)`; added warning logging for outer exception handlers. - `make_mock_lifecycle()` (mock_strategy_llm.py) and `_make_mock_lifecycle()` (llm_actors_coverage_steps.py): Added `resolve_actor_provider_model` to mock lifecycle services for test compatibility. - Added `@tdd_issue @tdd_issue_11254` tags to all new BDD scenarios related to namespace/name disambiguation in llm_actors_coverage, strategy_actor_llm, and plan_lifecycle_service_coverage_boost_r4 feature files. - Updated docs/CHANGELOG.md with the fix entry. ISSUES CLOSED: #11254	2026-05-22 20:42:25 +01:00
hurui200320	80c8636c4a	Revert "feat: add fallback to Anthropic Sonnet when OpenAI quota is exhausted" ... This reverts commit f5712787e0.	2026-04-17 18:00:47 +08:00
CoreRasurae	f5712787e0	feat: add fallback to Anthropic Sonnet when OpenAI quota is exhausted ... Implements graceful degradation for E2E robot integration tests that hit OpenAI 429 quota limit errors. Changes: - Add _is_quota_error() helper to detect quota-specific API errors (429, insufficient_quota, rate_limit) - Modify _execute_with_llm() in StrategyActor to catch quota errors and attempt fallback to Anthropic Haiku - Configure fallback provider as 'anthropic/claude-sonnet-4-20250514' - Add comprehensive logging for quota error detection and provider fallback - Add E2E test scenarios for quota fallback verification When quota errors occur on both OpenAI and Anthropic fallback, tests now fail with a clear message explaining that the test outcome cannot be verified when no LLM provider is available. This ensures CI/CD pipelines properly track which tests could not be executed due to quota constraints, rather than silently skipping them and creating false confidence in test coverage. This ensures CI/CD pipelines can complete E2E tests even when the primary provider (OpenAI) hits quota limits, improving pipeline reliability and reducing false negatives caused by provider-specific issues. 1. **Cache fallback_llm instance** - Instead of recreating the fallback LLM every time a quota error occurs, cache it as an instance variable (self._fallback_llm). This avoids unnecessary re-initialization overhead. 2. **Implement quota recovery logic** - Add intelligent recovery behavior: - Track last quota error timestamp (self._last_quota_error_time) - Track fallback mode state (self._using_fallback) - Once quota error detected, switch to fallback provider - Only attempt to recover primary provider every 5 minutes (_QUOTA_RECOVERY_INTERVAL) - This avoids hammering primary provider with repeated quota errors 3. **Add detailed recovery logging** - Log quota fallback transitions and recovery attempts to improve observability and debugging. Benefits: - Reduced latency: No redundant primary provider calls after quota error - Reduced overhead: Cached fallback LLM instance, no per-call recreation - Better observability: Clear logging of fallback mode entry/exit - Intelligent recovery: Automatic recovery attempt after 5-minute interval Updated tests: - M6 E2E Event Queue Via Plan Lifecycle Transitions - M6 E2E Hierarchical Decomposition Via Plan Tree - M6 E2E Full Autonomy Acceptance Flow Fixes: #10042	2026-04-17 03:56:35 +00:00
CoreRasurae	d3cb534caf	feat(plan): implement LLM-powered strategy actor ... Implement StrategyActor class for the plan strategize phase that uses an LLM to produce hierarchical execution strategies with dependencies, resource requirements, estimated complexity, and risk scores. Key components: - StrategyActor: Core actor with LLM prompt construction, response parsing (JSON and numbered-list fallback), and graceful degradation to StrategizeStubActor when no LLM provider is configured - StrategyAction/StrategyTree: Pydantic models for the hierarchical action tree with dependency links - validate_no_cycles(): Kahns algorithm (deque-based) for dependency graph cycle detection, raising PlanError on circular dependencies - build_strategy_prompt(): Context-aware prompt construction using definition_of_done, resources, project context, and ACMS analysis with XML-delimited user content sections for prompt injection hardening - parse_strategy_response(): Robust LLM output parsing with JSON extraction and numbered-list fallback - resolve_strategy_actor(): Integration point for the existing actor.default.strategy config key (CLEVERAGENTS_DEFAULT_STRATEGY_ACTOR) - Decision conversion producing strategy_choice Decision objects - build_decisions() preserves tree hierarchy via parent_id mapping, populates downstream_decision_ids from dependency edges, and validates plan_id Structural tree hierarchy (B2 review fix): - _build_tree infers parent_id from the dependency graph: each actions first resolved dependency becomes its structural parent. Actions with no dependencies fall back to the root. This produces hierarchical trees for agents plan tree rendering per spec Plan Decision Tree. Downstream decision tracking (B3 review fix): - build_decisions populates downstream_decision_ids from the strategy trees dependency edges using a pre-generated decision_id map so influence relationships between decisions are recorded per the spec Decision Record Structure. Post code-review hardening (PR #1175): - Broadened exception handling in execute() and ACMS retrieval to catch all LLM provider errors (openai, httpx, anthropic, etc.) with graceful fallback to stub mode (H1, H2) - Added warning log for unresolvable dependency references so dropped edges are visible in structured logs (H3) - Added XML-delimited user content sections and explicit data-only instructions in system prompt for prompt injection hardening (H4) - Switched prompt truncation to word-boundary-safe _truncate_at_word() for all prompt input sections (M1) - Fixed _parse_actor_name to preserve user-specified provider or model when only one segment is empty, instead of discarding both (M2) - Annotated _build_invariant_records as placeholder pending the Invariant Reconciliation Actor implementation (M5) - Documented resources/project_context params as future-wired through PlanExecutor.run_strategize() (M8) - Added docstring noting supersession relationship with LLMStrategizeActor in llm_actors.py (M9) - Added __all__ export definition (L2) - Improved validate_no_cycles docstring edge direction semantics (L7) - Cap JSON parse retry loop at _MAX_JSON_PARSE_RETRIES (10) Post second code-review hardening (PR #1175, review cycle 2): - Fixed _truncate_at_word docstring: documented max_chars >= 3 precondition for the result-length guarantee (R-H1) - Added warning log in build_decisions for unresolvable parent_id references, matching the existing _build_tree warning for unresolvable dependency references (R-H2) - Fixed _parse_actor_name to handle whitespace-only input by adding actor_name.strip() check alongside the emptiness check (R-M1) - Tightened ACMS scenario assertions from non-empty to expected count of 5 decisions (R-L3) - Added timeout=60s on_timeout=kill to all Robot test cases for consistency with project patterns (R-M5) Post third code-review hardening (PR #1175, review cycle 3): - Added _sanitize_xml_content() to escape XML special characters (<, >, &) in user content before embedding into XML-delimited prompt sections, preventing prompt injection via forged closing tags (spec Prompt Injection Mitigation) (CR3-M1) - Upgraded _try_parse_json() to multi-anchor retry: collects all [{ positions left-to-right and tries each as a candidate start, fixing false-start anchoring when LLM preamble contains [{ fragments before the real JSON array (CR3-M2) - Added _truncate_at_word() guard for max_chars < 3: returns a hard slice instead of word-boundary truncation when the ellipsis would exceed the limit (CR3-L2) - Changed _build_tree collision fallback key from -(idx+1) to -(1_000_000+idx) to eliminate theoretical collision with LLM-produced negative step numbers (CR3-L3) - Added forward-looking API docstring note to build_decisions() documenting that it is not yet wired into PlanExecutor and will be integrated once Decision persistence lands (CR3-M3) Post fourth code-review hardening (PR #1175, review cycle 4): - Fixed _try_parse_json per-anchor retry counter: reset retries=0 at the start of each anchor iteration so false-start [{ anchors in LLM preamble text no longer exhaust the retry budget for the correct anchor (CR4-B1) - Added known-limitations docstring to module header documenting missing decision types (resource_selection, subplan_spawn, invariant_enforced) as future work (CR4-D1) - Rewrote XML injection assertion in test to use regex extraction instead of fragile chained .split() calls that could IndexError on structural changes (CR4-T5) Post fifth code-review hardening (PR #1175, review cycle 5): - Added warning log in build_decisions for empty-string parent_id (distinct from None) so the silent fallback to root is visible in structured logs for debuggability (CR5-B1) - Added plan_id propagation assertion to build_decisions test scenarios verifying decision.plan_id matches the input (CR5-T1) - Added sequence_number monotonicity assertion verifying decision sequence_numbers are zero-indexed and monotonically increasing (CR5-T2) - Added _truncate_at_word boundary test for max_chars=3 (exactly ellipsis length) verifying correct "..." output (CR5-T3) - Tightened false-start anchor test from permissive len>=1 to specific description match "Sole real action" (CR5-T4) - Added word-boundary truncation test using space-separated input to exercise the rfind(" ") path under oversized DoD (CR5-T5) Post sixth code-review hardening (PR #1175, review cycle 6): - Added _MAX_INVARIANTS cap (100) for invariant list truncation in prompt to prevent token limit overflows, consistent with other prompt section caps (CR6-M4) - Added negative max_chars guard in _truncate_at_word returning empty string instead of slicing from end (CR6-M5) - Added global JSON parse attempt cap _MAX_GLOBAL_JSON_ATTEMPTS (50) across all anchors in _try_parse_json (CR6-L3) - Moved re import to module level in strategy_parsing.py per CONTRIBUTING import guidelines (CR6-L4) - Extracted _DEFAULT_DESCRIPTION constant to eliminate duplication between _default_action() and _build_tree() (CR6-L5) Post seventh code-review hardening (PR #1175, review cycle 7): - Decoupled _execute_stub from StrategizeStubActor._parse_steps private method by delegating to parse_strategy_response, removing cross-class private method dependency (CR7-M1) - Added ULID format validation on plan_id in execute() and build_decisions() for spec-consistent argument validation per §Plan glossary and CONTRIBUTING §Argument Validation (CR7-M2) - Constrained StrategyAction.estimated_complexity to Literal["low", "medium", "high"] at Pydantic model level per CONTRIBUTING §Type Safety (CR7-M5) - Documented XML-tag prompt boundary deviation from spec [USER_CONTENT_START]/[USER_CONTENT_END] markers with rationale for the more structured approach (CR7-M6) - Added _build_tree empty-input guard comment documenting orphaned root_id semantics (CR7-L1) - Added _truncate_at_word > 0 intent comment explaining why position-0 space is intentionally excluded (CR7-L2) - Added build_decisions context_snapshot future-work comment referencing spec §Decision Record Structure (CR7-L5) - Used enumerate() in _build_tree first loop for idiomatic Python (CR7-L7) - Fixed false-start anchor test (CR5-T4) broken by CR6-L3 global cap: reduced preamble fragments from 15 to 3 so total attempts stay within _MAX_GLOBAL_JSON_ATTEMPTS (CR7-T1) - Fixed test plan_ids containing non-Crockford-Base32 characters (L→K) to pass ULID format validation (CR7-T2) Tests: - 105 Behave BDD scenarios in features/strategy_actor_llm.feature adding: global JSON attempt cap exhaustion (CR7-L3), orphaned dependency edge silent drop (CR7-L4), non-ULID plan_id rejection in execute() and build_decisions() (CR7-M2) - 101 Behave BDD scenarios in features/strategy_actor_llm.feature including new scenarios for _truncate_at_word edge cases (L3), create_llm argument verification (L4), non-numeric step field fallback (L5), updated assertions for XML-delimited prompts and _parse_actor_name partial-segment preservation (M2), lifecycle exception fallback (R1), PydanticValidationError re-raise verification (R2), self-loop cycle detection (R3), whitespace-only actor name (R4), XML tag injection sanitisation (CR3-M1), preamble bracket fragment parsing (CR3-M2), _truncate_at_word sub-3 limit (CR3-L2), resolve_strategy_actor with both llm config and registry (CR3-L5), build_decisions unresolvable parent_id fallback (CR3-L7), XML injection in resources/project_context/acms_context fields (CR4-S1), ampersand escaping (CR4-S1d), false-start anchor retry budget (CR4-T3), non-sequential step edge specificity (CR4-T4), plan_id propagation (CR5-T1), sequence_number monotonicity (CR5-T2), max_chars=3 boundary (CR5-T3), false-start anchor specificity (CR5-T4), word-boundary truncation (CR5-T5), invariant prompt constraints (CR6-M2), invariant XML sanitisation (CR6-M3), invariant truncation cap (CR6-M4), negative max_chars (CR6-M5), and no-space truncation (CR6-L8) - 7 Robot Framework integration tests in robot/strategy_actor.robot - Mock LLM provider in features/mocks/mock_strategy_llm.py All nox stages pass: lint, typecheck, unit_tests (13789 scenarios), integration_tests (1863 passed). integration_tests (1863 passed, 2 pre-existing TDD failures unrelated to this change). ISSUES CLOSED: #828	2026-04-14 19:26:49 +00:00