HTTP/2 200 server: GitHub.com content-type: text/html; charset=utf-8 last-modified: Fri, 28 Jun 2024 08:22:08 GMT access-control-allow-origin: * strict-transport-security: max-age=31556952 etag: W/"667e72b0-2177" expires: Mon, 29 Dec 2025 19:13:16 GMT cache-control: max-age=600 content-encoding: gzip x-proxy-cache: MISS x-github-request-id: 2514:123DE:935E56:A550E2:6952D074 accept-ranges: bytes age: 0 date: Mon, 29 Dec 2025 19:03:16 GMT via: 1.1 varnish x-served-by: cache-bom-vanm7210070-BOM x-cache: MISS x-cache-hits: 0 x-timer: S1767034997.710893,VS0,VE210 vary: Accept-Encoding x-fastly-request-id: a75a3a7fc1a23422e334611ab64088bdb9d87ceb content-length: 2565

Auto Arena of LLMs: Automating LLM Evaluations with Agent Peer-battles and Committee Discussions

🤗 Leaderboard Paper Blog Github Debate Samples

We present Auto Arena of LLMs, a completely automated large language model (LLM) evaluation framework that comprehensively investigates an LLM’s capabilities by employing various LLM agents in peer-battles and committee discussions. On the models that are included in Chatbot Arena, we can recover human preference scores with 94.5% correlation, which exceeds all current benchmarks.