Arena Leaderboard Policy

Scope

Reviewed benchmark or evaluation-protocol identity; model results and leaderboard ranks are separate, unrepresented observations.

Boundary

Human preference is not factual correctness, safety or universal capability. Model listing/deprecation and changing votes prevent a static rank from becoming an entity attribute.

Benchmark comparability passport

Publisher

Arena Team

Version

last updated 2026-09-01

Task scope

Live community evaluation of eligible foundation models through randomized battles and human preference votes across Arena surfaces.

Metric contract

Leaderboard ratings are estimated from battle outcomes with a regression that reweights sampling; ranks and intervals depend on the current methodology and accumulated vote data.

Evaluation conditions

Result identity includes Arena surface/category, policy and methodology revision, model endpoint identity and availability, battle/vote window, filters, sampling/reweighting and data-cleaning rules.

Comparability boundary

The leaderboard is live and methodology changes are logged; a rank is not timeless and must retain its observation time, category, policy/method version, interval and vote window.

Contamination boundary

Live user prompts reduce dependence on one fixed public test set, but the policy does not establish absence of manipulation, selection effects, provider exposure or preference bias.

This passport describes a benchmark definition or protocol. It publishes no model score, rank or cross-version equivalence.

Source-qualified relations

1. No reviewed relation is asserted for this identity.

Atlas claims with an exact source route

1. No exact source-URL connection in r0056. No name match was substituted.