Methodology v0.1 · provisional

How Model Gauntlet scores the AI race.

The site publishes two deliberately separate views: an official point-in-time LMArena snapshot dated 2026-07-12, and 11 selected historical snapshots ending August 29, 2025. They use different methods and are never connected into one performance line.

Publication boundaries

One scored table. Two unscored context feeds.

A signal moves from source to published state only after its source, overlap, and sanity checks pass.

ArchiveValidatePublish
01

Capability

The current table selects the highest Arena Score for each official organization in one pinned text style-control snapshot.

Published point-in-time signal
02

Adoption

OpenRouter model count describes catalog inventory only. It is not adoption or usage, and it is not scored.

Context only
03

Release velocity

Matched Epoch notable-model records provide dated release context. They do not affect any score.

Context only

Frontier rule

Labs, not model SKUs.

For the current snapshot, the organization field comes from LMArena’s official dataset. We then select that organization’s highest Arena Score in the overall category. Historical slugs use a reviewed, fail-closed ownership map: ambiguous or third-party derivatives are excluded rather than guessed. 

frontier(organization, snapshot) = max(Arena Score in overall category)

Composite status

No published weights yet.

The homepage contains no composite leaderboard. The older seed artifact has null composite_score values and is not rendered as a scored product.

Any future composite needs a versioned formula, per-field provenance, licensed inputs, fallbacks, a change log, and human approval before it can appear on an indexed page.

Update cadence

Pinned snapshot now, reconciled tail later.

The historical chart contains 11 selected snapshots from 2023-05-08 through 2025-08-29. It crosses known LMArena method changes: Bradley-Terry replaced Elo on January 9, 2024; style control became the default on May 16, 2025; and frequency reweighting arrived July 23, 2025. The chart therefore describes published sampled ratings, not a normalized like-for-like trajectory. A live append is held because the legacy and current overlap differs materially; no threshold waiver is allowed.

Sources and licensing

Every source keeps its boundary.

  • LMArena leaderboard dataset for the current table, pinned to revision 4a6c22ada142f24563a2ecc5ceb1ce94ab8be019 and licensed CC BY 4.0.
  • Historical LMArena Space artifacts for the 11 selected legacy snapshots. Their source boundary and Apache-2.0 repository metadata are kept separate from the current dataset.
  • Epoch AI notable models for dated release context only.
  • OpenRouter model metadata for catalog inventory context only. Inventory is not adoption; token share is not published here.
  • Artificial Analysis data is never ingested or republished.

Method questions

Direct answers, no implied guarantees.

How often does the site update?

The official point-in-time table was published 2026-07-12. The separate historical view contains 11 selected snapshots through August 29, 2025. Neither is described as live.

What are the composite weights?

There is no published composite. OpenRouter catalog inventory and Epoch release records are context only; neither changes the Arena Score table.

Does Model Gauntlet use Artificial Analysis data?

No. Model Gauntlet does not ingest or republish Artificial Analysis data. Its public presentation may inform interface research only.