{"id":2461,"url":"https://git.cleverthis.com/cleveragents/cleveragents-core/pulls/10846","number":10846,"user":{"id":140,"login":"HAL9000","login_name":"","source_id":0,"full_name":"HAL 9000","email":"140+hal9000@noreply.git.cleverthis.com","avatar_url":"https://git.cleverthis.com/avatars/90c410a38fbfcfcda5409767088b8a78d7579d30aa917329e6ff7aafee943b97","html_url":"https://git.cleverthis.com/HAL9000","language":"","is_admin":false,"last_login":"0001-01-01T00:00:00Z","created":"2026-04-06T15:48:20Z","restricted":false,"active":false,"prohibit_login":false,"location":"","website":"https://cleverthis.com","description":"I am just your friendly AI coding assistant and bot.","visibility":"public","followers_count":0,"following_count":0,"starred_repos_count":0,"username":"HAL9000"},"title":"perf(ci): optimize benchmark-regression test suite to reduce CI execution time","body":"## Summary\n\n- Added `benchmark_regression_fast` nox session that excludes the three slowest benchmark suites (`IndexingScalingSuite`, `ContextAssemblyScalingSuite`, `ExecutionThroughputSuite`) from PR regression checks\n- Added `benchmark_regression` CI job to `ci.yml` using the fast session with a 20-minute timeout, triggered on every PR\n- Added full `benchmark_regression` run to the nightly quality workflow so the complete suite still runs on a schedule\n- Documented the excluded suites and their timeout characteristics in each benchmark file\n\n## Problem\n\nThe `benchmark-regression` test suite was taking over 50 minutes in CI, primarily due to three benchmark suites with very high timeouts:\n\n1. `IndexingScalingSuite` (`large_project_scaling_bench`) — 600 s timeout, runs `walk_and_index` at up to 100K files\n2. `ContextAssemblyScalingSuite` (`context_assembly_scaling_bench`) — 300 s timeout, assembles ACMS context at up to 10K fragments\n3. `ExecutionThroughputSuite` (`execution_throughput_bench`) — 300 s timeout, executes up to 100 sequential plans\n\n## Solution\n\nThe fast PR subset excludes these three suites via ASV `--bench` regex pattern, targeting under 15 minutes wall-clock time. The full suite continues to run nightly via the `benchmark_regression` session.\n\nCloses #1668\n\nThis PR blocks issue #1668\n\n---\n**Automated by CleverAgents Bot**\nSupervisor: Implementation | Agent: implementation-worker","labels":[{"id":1663,"name":"auto/needs-reevaluation","exclusive":false,"is_archived":false,"color":"cccccc","description":"Controller deferred this PR; awaiting Phase 6+ scope-evaluator or operator re-enablement.","url":"https://git.cleverthis.com/api/v1/repos/cleveragents/cleveragents-core/labels/1663"},{"id":1661,"name":"controller-managed","exclusive":false,"is_archived":false,"color":"7057ff","description":"Auto-agents controller manages this PR/issue (see tools/controller/deploy/RUNBOOK.md). Remove this label to abandon controller management.","url":"https://git.cleverthis.com/api/v1/repos/cleveragents/cleveragents-core/labels/1661"}],"milestone":{"id":131,"title":"v3.8.0","description":"# v3.8.0 — M9: Server Implementation\n\n**Goal**: Implement the CleverAgents server and all server-dependent capabilities using the A2A (Agent-to-Agent) protocol standard.\n\n**Scope:**\n- A2A JSON-RPC 2.0 wire format and _cleveragents/ extension method routing\n- A2A stdio transport (local mode — agent as subprocess)\n- A2A HTTP transport (server mode — single JSON-RPC endpoint)\n- LangGraph Platform RemoteGraph integration for server-side actor execution\n- Server application structure (FastAPI + A2A SDK)\n- Authentication and authorization (API tokens, team RBAC)\n- Entity sync via _cleveragents/sync/* extension methods\n- Multi-device experience, shared namespaces, PostgreSQL backend\n- Server deployment (Docker + Kubernetes + Helm)\n\n**Key ADRs:** ADR-047 (A2A Standard Adoption), ADR-048 (Server Application Architecture)\n\n**No deadline assigned.** Development effort should focus on M1-M6 milestones first.","state":"open","open_issues":357,"closed_issues":185,"created_at":"2026-03-11T18:08:43Z","updated_at":"2026-06-18T12:01:59Z","closed_at":null,"due_on":null},"assignee":null,"assignees":[],"requested_reviewers":[{"id":141,"login":"HAL9001","login_name":"","source_id":0,"full_name":"HAL9001","email":"141+hal9001@noreply.git.cleverthis.com","avatar_url":"https://git.cleverthis.com/avatars/8fa8ce6c4c2daadbfe10d30373bf14a91a09d26693cbc72a440a82518c77658e","html_url":"https://git.cleverthis.com/HAL9001","language":"","is_admin":false,"last_login":"0001-01-01T00:00:00Z","created":"2026-04-10T20:13:03Z","restricted":false,"active":false,"prohibit_login":false,"location":"","website":"https://CleverThis.com","description":"","visibility":"public","followers_count":0,"following_count":0,"starred_repos_count":0,"username":"HAL9001"}],"requested_reviewers_teams":[],"state":"open","draft":false,"is_locked":false,"comments":8,"review_comments":2,"additions":113,"deletions":9,"changed_files":7,"html_url":"https://git.cleverthis.com/cleveragents/cleveragents-core/pulls/10846","diff_url":"https://git.cleverthis.com/cleveragents/cleveragents-core/pulls/10846.diff","patch_url":"https://git.cleverthis.com/cleveragents/cleveragents-core/pulls/10846.patch","mergeable":false,"merged":false,"merged_at":null,"merge_commit_sha":null,"merged_by":null,"allow_maintainer_edit":false,"base":{"label":"master","ref":"master","sha":"018a316768386c8ea17f9b411fd9c1a466925ad6","repo_id":271,"repo":{"id":271,"owner":{"id":124,"login":"cleveragents","login_name":"","source_id":0,"full_name":"","email":"","avatar_url":"https://git.cleverthis.com/avatars/a30617c9a40165a97294bd5de610892e","html_url":"https://git.cleverthis.com/cleveragents","language":"","is_admin":false,"last_login":"0001-01-01T00:00:00Z","created":"2025-06-28T18:50:27Z","restricted":false,"active":false,"prohibit_login":false,"location":"","website":"","description":"","visibility":"public","followers_count":0,"following_count":0,"starred_repos_count":0,"username":"cleveragents"},"name":"cleveragents-core","full_name":"cleveragents/cleveragents-core","description":"test","empty":false,"private":false,"fork":false,"template":false,"mirror":false,"size":86515,"language":"","languages_url":"https://git.cleverthis.com/api/v1/repos/cleveragents/cleveragents-core/languages","html_url":"https://git.cleverthis.com/cleveragents/cleveragents-core","url":"https://git.cleverthis.com/api/v1/repos/cleveragents/cleveragents-core","link":"","ssh_url":"git@git.cleverthis.com:cleveragents/cleveragents-core.git","clone_url":"https://git.cleverthis.com/cleveragents/cleveragents-core.git","original_url":"","website":"","stars_count":0,"forks_count":3,"watchers_count":4,"branch_count":1418,"open_issues_count":5979,"open_pr_counter":115,"release_counter":0,"default_branch":"master","archived":false,"created_at":"2025-06-28T18:50:47Z","updated_at":"2026-06-19T07:59:07Z","archived_at":"1970-01-01T00:00:00Z","permissions":{"admin":false,"push":false,"pull":true},"has_code":true,"has_issues":true,"internal_tracker":{"enable_time_tracker":false,"allow_only_contributors_to_track_time":false,"enable_issue_dependencies":true},"has_wiki":true,"has_pull_requests":true,"has_projects":true,"projects_mode":"all","has_releases":true,"has_packages":true,"has_actions":true,"ignore_whitespace_conflicts":false,"allow_merge_commits":true,"allow_rebase":true,"allow_rebase_explicit":true,"allow_squash_merge":true,"allow_fast_forward_only_merge":true,"allow_rebase_update":true,"allow_manual_merge":false,"autodetect_manual_merge":false,"default_delete_branch_after_merge":false,"default_merge_style":"merge","default_allow_maintainer_edit":false,"avatar_url":"","internal":false,"mirror_interval":"","object_format_name":"sha1","mirror_updated":"0001-01-01T00:00:00Z","topics":[],"licenses":[]}},"head":{"label":"test/ci-execution-time-optimize-benchmark-regression","ref":"test/ci-execution-time-optimize-benchmark-regression","sha":"398186a23ac2b3392644abe53964f0d6bda0b3f5","repo_id":271,"repo":{"id":271,"owner":{"id":124,"login":"cleveragents","login_name":"","source_id":0,"full_name":"","email":"","avatar_url":"https://git.cleverthis.com/avatars/a30617c9a40165a97294bd5de610892e","html_url":"https://git.cleverthis.com/cleveragents","language":"","is_admin":false,"last_login":"0001-01-01T00:00:00Z","created":"2025-06-28T18:50:27Z","restricted":false,"active":false,"prohibit_login":false,"location":"","website":"","description":"","visibility":"public","followers_count":0,"following_count":0,"starred_repos_count":0,"username":"cleveragents"},"name":"cleveragents-core","full_name":"cleveragents/cleveragents-core","description":"test","empty":false,"private":false,"fork":false,"template":false,"mirror":false,"size":86515,"language":"","languages_url":"https://git.cleverthis.com/api/v1/repos/cleveragents/cleveragents-core/languages","html_url":"https://git.cleverthis.com/cleveragents/cleveragents-core","url":"https://git.cleverthis.com/api/v1/repos/cleveragents/cleveragents-core","link":"","ssh_url":"git@git.cleverthis.com:cleveragents/cleveragents-core.git","clone_url":"https://git.cleverthis.com/cleveragents/cleveragents-core.git","original_url":"","website":"","stars_count":0,"forks_count":3,"watchers_count":4,"branch_count":1418,"open_issues_count":5979,"open_pr_counter":115,"release_counter":0,"default_branch":"master","archived":false,"created_at":"2025-06-28T18:50:47Z","updated_at":"2026-06-19T07:59:07Z","archived_at":"1970-01-01T00:00:00Z","permissions":{"admin":false,"push":false,"pull":true},"has_code":true,"has_issues":true,"internal_tracker":{"enable_time_tracker":false,"allow_only_contributors_to_track_time":false,"enable_issue_dependencies":true},"has_wiki":true,"has_pull_requests":true,"has_projects":true,"projects_mode":"all","has_releases":true,"has_packages":true,"has_actions":true,"ignore_whitespace_conflicts":false,"allow_merge_commits":true,"allow_rebase":true,"allow_rebase_explicit":true,"allow_squash_merge":true,"allow_fast_forward_only_merge":true,"allow_rebase_update":true,"allow_manual_merge":false,"autodetect_manual_merge":false,"default_delete_branch_after_merge":false,"default_merge_style":"merge","default_allow_maintainer_edit":false,"avatar_url":"","internal":false,"mirror_interval":"","object_format_name":"sha1","mirror_updated":"0001-01-01T00:00:00Z","topics":[],"licenses":[]}},"merge_base":"f829e68911adad00f9d278f4ace30b7b5c2bd813","due_date":null,"created_at":"2026-04-23T17:50:01Z","updated_at":"2026-06-18T15:08:52Z","closed_at":null,"pin_order":0,"content_version":0}