Talk:Language model benchmark
Add topic | This article is rated List-class on Wikipedia's content assessment scale. It is of interest to the following WikiProjects: | ||||||||||
| |||||||||||
comment
[edit source]If some of the benchmarks look weirdly obscure, then apologies. My criteria is simply: If a frontier model is advertised by showing how good it is on *this* or *that* benchmark, then I will put that benchmark in. For example, today I put in "Vibe-Eval", not because it is particularly interesting (I think it is not), but simply because the latest Google Gemini 2.5 (2025-06-05) advertised its ability on Vibe-Eval, so I had to put it in. pony in a strange land (talk) 20:58, 7 June 2025 (UTC)
MLCommons
[edit source]A coherent text on the subject, published in 2024 and added by me to Sources, describes something called MLCommons as the only benchmark standardization game in town. I do not know anything about it, but here is a questions to the one who know: is the omission of MLCommons in this long list deliberate (e.g., due to the bias of the source I have used) or accidental? Викидим (talk) 19:37, 12 September 2025 (UTC)
Proposal to restore external links
[edit source]As part of the AI-related presumptive removal process, this article lost an == External links == section that I had added. Rather than seeking restoration of the "Benchmark performance" section I would like editors to consider restoring only the two external links:
I believe these are useful external resources for readers. The first is the official website of the ARC Prize benchmark discussed in the article, while the second provides a comprehensive and regularly maintained index of language-model benchmark results that complements rather than duplicates the encyclopedic content. Could editors review whether these two links satisfy the external links guideline and, if so, whether they should be restored independently of the material that was removed as part of the AI cleanup? @Fifteen thousand two hundred twenty four: Ryoske (talk) 07:44, 29 June 2026 (UTC)
- I've restored LLM Stats in Special:Diff/1361651091 with a shorter description. Didn't re-add the ARC Prize link since it's not a very general target and it's already linked in the article where relevant as a reference (not excluding based on llmprv, it just doesn't seem like a good external link target) fifteen thousand two hundred twenty four (talk) 08:15, 29 June 2026 (UTC)