Metatron Intelligence
Today: the five AI headlines of the day → and the AI Wiki
What an entity record is How to read this page

A named actor, followed across desks

An organisation, model, chip or programme, with the statements attached to it, so that the same actor can be tracked without re-reading every domain.

Statements carry their own sources

Each statement on this page is bound to a source and a date. The entity page collects them; it does not average them into a verdict.

Absence is not evidence

An entity with few statements has been observed less, not necessarily done less. Read the count as coverage, never as significance.

New to this publication?

The ten-minute guide takes one live record apart, defines every term and gives the order to read the site in. Start here →

ENT-b2bceffe-2ce0-475c-b161-529c13553b46 · benchmark · software issue resolution benchmark subset

SWE-bench Verified

Reviewed identifier SWE-bench_Verified · current status intentionally unknown

Scope

Reviewed benchmark or evaluation-protocol identity; model results and leaderboard ranks are separate, unrepresented observations.

Boundary

Verified means the subset was reviewed for solvability; it is not verification of a submitted model result, software quality or production engineering ability.

Benchmark comparability passport

Publisher
SWE-bench
Version
Verified subset; source revision 02e7a74f
Task scope
Subset of 500 software-repository problems confirmed by software engineers as solvable.
Metric contract
Per-instance resolved/not-resolved outcome produced by the selected SWE-bench evaluation harness; no cross-harness aggregate is represented.
Evaluation conditions
Result identity requires exact dataset revision, task repository state, prediction patch, container image/harness version, run ID and grading logs.
Comparability boundary
Do not compare results across dataset, task-repository, image, harness or grading revisions without an explicit bridge; cached run IDs can reuse earlier results even when predictions change.
Contamination boundary
The selected source does not establish absence of issue, patch or repository exposure in model training or tool retrieval.

This passport describes a benchmark definition or protocol. It publishes no model score, rank or cross-version equivalence.

Source-qualified relations

  1. No reviewed relation is asserted for this identity.

Atlas claims with an exact source route

  1. No exact source-URL connection in r0056. No name match was substituted.

Every displayed attribute links to source-qualified statement IDs. Relations are explicit and source-qualified. Identity does not imply ownership, legal status, present availability or equivalence with a mutable alias.