diff --git a/features/budget_enforcement_plan_executor.feature b/features/budget_enforcement_plan_executor.feature new file mode 100644 index 000000000..17a066fdc --- /dev/null +++ b/features/budget_enforcement_plan_executor.feature @@ -0,0 +1,164 @@ +Feature: Budget enforcement in PlanExecutor + As a platform operator + I want PlanExecutor to halt execution when budget limits are exceeded + So that autonomous agents cannot overspend configured limits + + # ------------------------------------------------------------------ + # BudgetExceededError and PlanBudgetExceededError exceptions + # ------------------------------------------------------------------ + + Scenario: BudgetExceededError has correct attributes + When I create a BudgetExceededError with plan_id "plan-1" budget_type "daily" used 5.0 limit 3.0 + Then the BudgetExceededError plan_id should be "plan-1" + And the BudgetExceededError budget_type should be "daily" + And the BudgetExceededError used should be 5.0 + And the BudgetExceededError limit should be 3.0 + And the BudgetExceededError message should contain "budget" + + Scenario: PlanBudgetExceededError has correct attributes + When I create a PlanBudgetExceededError with plan_id "plan-2" used 10.0 limit 5.0 + Then the PlanBudgetExceededError plan_id should be "plan-2" + And the PlanBudgetExceededError used should be 10.0 + And the PlanBudgetExceededError limit should be 5.0 + And the PlanBudgetExceededError message should contain "budget" + + Scenario: BudgetExceededError is a subclass of PlanError + When I create a BudgetExceededError with plan_id "plan-3" budget_type "session" used 1.0 limit 0.5 + Then the BudgetExceededError should be an instance of PlanError + + Scenario: PlanBudgetExceededError is a subclass of PlanError + When I create a PlanBudgetExceededError with plan_id "plan-4" used 2.0 limit 1.0 + Then the PlanBudgetExceededError should be an instance of PlanError + + # ------------------------------------------------------------------ + # PlanExecutor budget enforcement - no cost tracker (no-op) + # ------------------------------------------------------------------ + + Scenario: PlanExecutor without cost_tracker does not check budget + Given a budget enforcement PlanExecutor without cost_tracker + And a budget enforcement plan in Execute-Queued state + When I run budget enforcement execute + Then the budget enforcement execute should succeed without budget error + + # ------------------------------------------------------------------ + # PlanExecutor budget enforcement - plan budget exceeded + # ------------------------------------------------------------------ + + Scenario: PlanExecutor halts with PlanBudgetExceededError when plan budget exceeded + Given a budget enforcement PlanExecutor with plan budget 0.0001 + And a budget enforcement plan in Execute-Queued state + And the cost_metadata has total_cost 0.001 + When I run budget enforcement execute expecting budget error + Then a PlanBudgetExceededError should be raised + And the PlanBudgetExceededError plan_id should match the plan + + Scenario: PlanExecutor saves plan state before halting on plan budget exceeded + Given a budget enforcement PlanExecutor with plan budget 0.0001 + And a budget enforcement plan in Execute-Queued state + And the cost_metadata has total_cost 0.001 + When I run budget enforcement execute expecting budget error + Then the lifecycle _commit_plan should have been called with budget_halt details + + # ------------------------------------------------------------------ + # PlanExecutor budget enforcement - daily budget exceeded + # ------------------------------------------------------------------ + + Scenario: PlanExecutor halts with BudgetExceededError when daily budget exceeded + Given a budget enforcement PlanExecutor with daily budget 0.0001 + And a budget enforcement plan in Execute-Queued state + And the daily spend is 0.001 + When I run budget enforcement execute expecting budget error + Then a BudgetExceededError should be raised + And the BudgetExceededError budget_type should be "daily" + + Scenario: PlanExecutor saves plan state before halting on daily budget exceeded + Given a budget enforcement PlanExecutor with daily budget 0.0001 + And a budget enforcement plan in Execute-Queued state + And the daily spend is 0.001 + When I run budget enforcement execute expecting budget error + Then the lifecycle _commit_plan should have been called with budget_halt details + + # ------------------------------------------------------------------ + # PlanExecutor budget enforcement - within budget (no halt) + # ------------------------------------------------------------------ + + Scenario: PlanExecutor continues when plan budget is not exceeded + Given a budget enforcement PlanExecutor with plan budget 100.0 + And a budget enforcement plan in Execute-Queued state + And the cost_metadata has total_cost 0.001 + When I run budget enforcement execute + Then the budget enforcement execute should succeed without budget error + + Scenario: PlanExecutor continues when daily budget is not exceeded + Given a budget enforcement PlanExecutor with daily budget 100.0 + And a budget enforcement plan in Execute-Queued state + And the daily spend is 0.001 + When I run budget enforcement execute + Then the budget enforcement execute should succeed without budget error + + # ------------------------------------------------------------------ + # AutomationProfile budget fields + # ------------------------------------------------------------------ + + Scenario: AutomationProfile has budget_per_plan field + When I create an AutomationProfile with budget_per_plan 10.0 + Then the AutomationProfile budget_per_plan should be 10.0 + + Scenario: AutomationProfile has budget_per_session field + When I create an AutomationProfile with budget_per_session 50.0 + Then the AutomationProfile budget_per_session should be 50.0 + + Scenario: AutomationProfile budget_per_plan defaults to None + When I create an AutomationProfile with default budget fields + Then the AutomationProfile budget_per_plan should be None + And the AutomationProfile budget_per_session should be None + + Scenario: AutomationProfile rejects negative budget_per_plan + When I try to create an AutomationProfile with budget_per_plan -1.0 + Then a budget enforcement validation error should be raised + + Scenario: AutomationProfile rejects negative budget_per_session + When I try to create an AutomationProfile with budget_per_session -1.0 + Then a budget enforcement validation error should be raised + + # ------------------------------------------------------------------ + # _check_budget method - direct unit tests + # ------------------------------------------------------------------ + + Scenario: _check_budget is a no-op when cost_tracker is None + Given a budget enforcement PlanExecutor without cost_tracker + When I call _check_budget directly with plan_id "test-plan" + Then no budget exception should be raised + + Scenario: _check_budget raises PlanBudgetExceededError when plan budget exceeded + Given a budget enforcement PlanExecutor with plan budget 0.0001 + And the cost_metadata has total_cost 0.001 + When I call _check_budget directly with plan_id "test-plan" + Then a PlanBudgetExceededError should be raised + + Scenario: _check_budget raises BudgetExceededError when daily budget exceeded + Given a budget enforcement PlanExecutor with daily budget 0.0001 + And the daily spend is 0.001 + When I call _check_budget directly with plan_id "test-plan" + Then a BudgetExceededError should be raised + + Scenario: _check_budget creates CostMetadata when none provided + Given a budget enforcement PlanExecutor with plan budget 100.0 and no cost_metadata + When I call _check_budget directly with plan_id "test-plan" + Then no budget exception should be raised + + # ------------------------------------------------------------------ + # _save_plan_state_on_budget_halt - graceful halt + # ------------------------------------------------------------------ + + Scenario: _save_plan_state_on_budget_halt persists budget details to plan + Given a budget enforcement PlanExecutor without cost_tracker + When I call _save_plan_state_on_budget_halt with plan_id "halt-plan" budget_type "plan" used 5.0 limit 3.0 + Then the lifecycle _commit_plan should have been called + And the plan error_details should contain budget_halt true + And the plan error_details should contain budget_type "plan" + + Scenario: _save_plan_state_on_budget_halt is non-fatal on lifecycle error + Given a budget enforcement PlanExecutor with failing lifecycle + When I call _save_plan_state_on_budget_halt with plan_id "halt-plan" budget_type "daily" used 1.0 limit 0.5 + Then no exception should be raised from _save_plan_state_on_budget_halt diff --git a/features/steps/tui_multi_session_tabs_steps.py b/features/steps/tui_multi_session_tabs_steps.py new file mode 100644 index 000000000..5187ed9fa --- /dev/null +++ b/features/steps/tui_multi_session_tabs_steps.py @@ -0,0 +1,287 @@ +"""Step definitions for TUI multi-session tabs feature.""" + +from __future__ import annotations + +from datetime import datetime + +from behave import given, then, when + +from cleveragents.tui.app import SessionView, _TextualCleverAgentsTuiApp +from cleveragents.tui.persona.registry import PersonaRegistry +from cleveragents.tui.persona.state import PersonaState + + +class MockCommandRouter: + """Mock command router for testing.""" + + def handle(self, raw: str, *, session_id: str) -> str: + """Mock command handler.""" + return f"Mock response for {raw} in session {session_id}" + + +@given("a TUI app is initialized with multi-session support") +def step_init_tui_app(context: object) -> None: + """Initialize a TUI app with multi-session support.""" + context.registry = PersonaRegistry() # type: ignore + context.persona_state = PersonaState(registry=context.registry) # type: ignore + context.router = MockCommandRouter() # type: ignore + # Note: We can't instantiate _TextualCleverAgentsTuiApp directly without Textual + # So we'll test the session management logic separately + + +@when("the TUI app is created") +def step_create_tui_app(context: object) -> None: + """Create a TUI app instance.""" + # Create a mock app with session management + context.app = type("MockApp", (), {})() # type: ignore + context.app._sessions = [ # type: ignore + SessionView( + session_id="default", + transcript=[], + name="Default", + created_at=datetime.utcnow().isoformat(), + ) + ] + context.app._active_session_index = 0 # type: ignore + + +@then("the app should have exactly {count:d} session") +def step_check_session_count(context: object, count: int) -> None: + """Check the number of sessions.""" + assert len(context.app._sessions) == count # type: ignore + + +@then("the active session should have session_id {session_id}") +def step_check_active_session_id(context: object, session_id: str) -> None: + """Check the active session ID.""" + active = context.app._sessions[context.app._active_session_index] # type: ignore + assert active.session_id == session_id + + +@then("the active session should have name {name}") +def step_check_active_session_name(context: object, name: str) -> None: + """Check the active session name.""" + active = context.app._sessions[context.app._active_session_index] # type: ignore + assert active.name == name + + +@when("I create a new session with name {name}") +def step_create_session(context: object, name: str) -> None: + """Create a new session.""" + import uuid + + session_id = str(uuid.uuid4())[:8] + new_session = SessionView( + session_id=session_id, + transcript=[], + name=name, + created_at=datetime.utcnow().isoformat(), + ) + context.app._sessions.append(new_session) # type: ignore + context.app._active_session_index = len(context.app._sessions) - 1 # type: ignore + + +@then("the new session should have an independent session_id") +def step_check_new_session_id(context: object) -> None: + """Check that the new session has a unique ID.""" + sessions = context.app._sessions # type: ignore + session_ids = [s.session_id for s in sessions] + assert len(session_ids) == len(set(session_ids)) # All unique + + +@given("the TUI app has {count:d} session") +def step_setup_sessions(context: object, count: int) -> None: + """Set up the TUI app with a specific number of sessions.""" + context.app = type("MockApp", (), {})() # type: ignore + context.app._sessions = [] # type: ignore + for i in range(count): + if i == 0: + session_id = "default" + name = "Default" + else: + import uuid + + session_id = str(uuid.uuid4())[:8] + name = f"Session {i + 1}" + session = SessionView( + session_id=session_id, + transcript=[], + name=name, + created_at=datetime.utcnow().isoformat(), + ) + context.app._sessions.append(session) # type: ignore + context.app._active_session_index = 0 # type: ignore + + +@given("the first session has session_id {session_id}") +def step_check_first_session_id(context: object, session_id: str) -> None: + """Verify the first session has the expected ID.""" + assert context.app._sessions[0].session_id == session_id # type: ignore + + +@given("the second session has session_id {session_id}") +def step_check_second_session_id(context: object, session_id: str) -> None: + """Verify the second session has the expected ID.""" + assert context.app._sessions[1].session_id == session_id # type: ignore + + +@when("I switch to session {session_id}") +def step_switch_session(context: object, session_id: str) -> None: + """Switch to a specific session.""" + for idx, session in enumerate(context.app._sessions): # type: ignore + if session.session_id == session_id: + context.app._active_session_index = idx # type: ignore + return + raise ValueError(f"Session {session_id} not found") + + +@when("I close the session with session_id {session_id}") +def step_close_session(context: object, session_id: str) -> None: + """Close a session.""" + if len(context.app._sessions) <= 1: # type: ignore + context.close_failed = True # type: ignore + return + for idx, session in enumerate(context.app._sessions): # type: ignore + if session.session_id == session_id: + context.app._sessions.pop(idx) # type: ignore + if context.app._active_session_index >= len(context.app._sessions): # type: ignore + context.app._active_session_index = len(context.app._sessions) - 1 # type: ignore + context.close_failed = False # type: ignore + return + raise ValueError(f"Session {session_id} not found") + + +@when("I try to close the session with session_id {session_id}") +def step_try_close_session(context: object, session_id: str) -> None: + """Try to close a session (may fail).""" + context.close_failed = False # type: ignore + if len(context.app._sessions) <= 1: # type: ignore + context.close_failed = True # type: ignore + return + for idx, session in enumerate(context.app._sessions): # type: ignore + if session.session_id == session_id: + context.app._sessions.pop(idx) # type: ignore + if context.app._active_session_index >= len(context.app._sessions): # type: ignore + context.app._active_session_index = len(context.app._sessions) - 1 # type: ignore + return + + +@then("the close operation should fail") +def step_check_close_failed(context: object) -> None: + """Check that the close operation failed.""" + assert context.close_failed # type: ignore + + +@when("I rename the session to {new_name}") +def step_rename_session(context: object, new_name: str) -> None: + """Rename the active session.""" + active = context.app._sessions[context.app._active_session_index] # type: ignore + active.name = new_name + + +@given("the active session has name {name}") +def step_check_active_session_has_name(context: object, name: str) -> None: + """Verify the active session has a specific name.""" + active = context.app._sessions[context.app._active_session_index] # type: ignore + assert active.name == name + + +@given("the first session is active") +def step_first_session_active(context: object) -> None: + """Make the first session active.""" + context.app._active_session_index = 0 # type: ignore + + +@when("I set persona {persona_name} for the first session") +def step_set_persona_first(context: object, persona_name: str) -> None: + """Set persona for the first session.""" + session_id = context.app._sessions[0].session_id # type: ignore + context.persona_state.active_by_session[session_id] = persona_name # type: ignore + + +@when("I set persona {persona_name} for the second session") +def step_set_persona_second(context: object, persona_name: str) -> None: + """Set persona for the second session.""" + session_id = context.app._sessions[1].session_id # type: ignore + context.persona_state.active_by_session[session_id] = persona_name # type: ignore + + +@when("I switch back to the first session") +def step_switch_back_to_first(context: object) -> None: + """Switch back to the first session.""" + context.app._active_session_index = 0 # type: ignore + + +@then("the first session should have active persona {persona_name}") +def step_check_first_session_persona(context: object, persona_name: str) -> None: + """Check the first session's active persona.""" + session_id = context.app._sessions[0].session_id # type: ignore + assert context.persona_state.active_by_session.get(session_id) == persona_name # type: ignore + + +@then("the second session should have active persona {persona_name}") +def step_check_second_session_persona(context: object, persona_name: str) -> None: + """Check the second session's active persona.""" + session_id = context.app._sessions[1].session_id # type: ignore + assert context.persona_state.active_by_session.get(session_id) == persona_name # type: ignore + + +@when("I add message {message} to the first session") +def step_add_message_first(context: object, message: str) -> None: + """Add a message to the first session.""" + context.app._sessions[0].transcript.append(message) # type: ignore + + +@when("I add message {message} to the second session") +def step_add_message_second(context: object, message: str) -> None: + """Add a message to the second session.""" + context.app._sessions[1].transcript.append(message) # type: ignore + + +@then("the first session transcript should contain {message}") +def step_check_first_transcript_contains(context: object, message: str) -> None: + """Check that the first session transcript contains a message.""" + assert message in context.app._sessions[0].transcript # type: ignore + + +@then("the first session transcript should not contain {message}") +def step_check_first_transcript_not_contains(context: object, message: str) -> None: + """Check that the first session transcript does not contain a message.""" + assert message not in context.app._sessions[0].transcript # type: ignore + + +@then("the second session transcript should contain {message}") +def step_check_second_transcript_contains(context: object, message: str) -> None: + """Check that the second session transcript contains a message.""" + assert message in context.app._sessions[1].transcript # type: ignore + + +@then("the second session transcript should not contain {message}") +def step_check_second_transcript_not_contains(context: object, message: str) -> None: + """Check that the second session transcript does not contain a message.""" + assert message not in context.app._sessions[1].transcript # type: ignore + + +@when("I create a new session") +def step_create_new_session(context: object) -> None: + """Create a new session.""" + import uuid + + session_id = str(uuid.uuid4())[:8] + new_session = SessionView( + session_id=session_id, + transcript=[], + name=f"Session {len(context.app._sessions) + 1}", # type: ignore + created_at=datetime.utcnow().isoformat(), + ) + context.app._sessions.append(new_session) # type: ignore + context.app._active_session_index = len(context.app._sessions) - 1 # type: ignore + context.new_session = new_session # type: ignore + + +@then("the new session should have a created_at timestamp in ISO format") +def step_check_created_at_timestamp(context: object) -> None: + """Check that the new session has a valid ISO format timestamp.""" + timestamp = context.new_session.created_at # type: ignore + # Try to parse it as ISO format + datetime.fromisoformat(timestamp) diff --git a/features/tui_multi_session_tabs.feature b/features/tui_multi_session_tabs.feature new file mode 100644 index 000000000..5edb213a0 --- /dev/null +++ b/features/tui_multi_session_tabs.feature @@ -0,0 +1,72 @@ +Feature: TUI Multi-Session Tabs with Independent A2A Bindings + The TUI supports multiple session tabs, each with independent A2A bindings, + persona selection, and conversation history. + + Background: + Given a TUI app is initialized with multi-session support + + Scenario: TUI starts with a default session + When the TUI app is created + Then the app should have exactly 1 session + And the active session should have session_id "default" + And the active session should have name "Default" + + Scenario: Create a new session + Given the TUI app has 1 session + When I create a new session with name "Session 2" + Then the app should have exactly 2 sessions + And the active session should have name "Session 2" + And the new session should have an independent session_id + + Scenario: Switch between sessions + Given the TUI app has 2 sessions + And the first session has session_id "default" + And the second session has session_id "sess-2" + When I switch to session "default" + Then the active session should have session_id "default" + When I switch to session "sess-2" + Then the active session should have session_id "sess-2" + + Scenario: Close a session + Given the TUI app has 2 sessions + When I close the session with session_id "sess-2" + Then the app should have exactly 1 session + And the active session should have session_id "default" + + Scenario: Cannot close the last session + Given the TUI app has 1 session + When I try to close the session with session_id "default" + Then the close operation should fail + And the app should still have exactly 1 session + + Scenario: Rename a session + Given the TUI app has 1 session + And the active session has name "Default" + When I rename the session to "My Session" + Then the active session should have name "My Session" + + Scenario: Each session has independent persona tracking + Given the TUI app has 2 sessions + And the first session is active + When I set persona "analyst" for the first session + And I switch to the second session + And I set persona "coder" for the second session + And I switch back to the first session + Then the first session should have active persona "analyst" + When I switch to the second session + Then the second session should have active persona "coder" + + Scenario: Each session has independent transcript + Given the TUI app has 2 sessions + And the first session is active + When I add message "Hello from session 1" to the first session + And I switch to the second session + And I add message "Hello from session 2" to the second session + Then the first session transcript should contain "Hello from session 1" + And the first session transcript should not contain "Hello from session 2" + And the second session transcript should contain "Hello from session 2" + And the second session transcript should not contain "Hello from session 1" + + Scenario: Session creation includes timestamp + When I create a new session + Then the new session should have a created_at timestamp in ISO format