It's interesting that it seems to the non thinking variant has actually regressed on a quite few benchmarks compared to flash-2.0. They seem to be prioritizing coding above all else. Even the thinking variant only has marginal gains on non coding.
Our table parsing benchmarking has flash-2.0 at 0.84, flash-2.5-lite at 0.80 (non-thinking), flash-2.5-lite at 0.80 (thinking). Kind of unfortunate to see.
This makes sense though, right? Flash-Lite is intended to be weaker than Flash - the comparisons should be flash-2.0 vs flash-2.5 and flash-lite-2.0 to flash-lite-2.5.
hyuuu · 14h ago
does the lite version have a faster token output? or time to first token?
mortsnort · 15h ago
Big update, they removed _preview from the model name.
AbuAssar · 16h ago
why not just call it Gemini 2.5 Lite, i.e why flash moniker is necessary?
Workaccount2 · 16h ago
Because it is technically the replacement for Gemini Flash non-thinking.
Our table parsing benchmarking has flash-2.0 at 0.84, flash-2.5-lite at 0.80 (non-thinking), flash-2.5-lite at 0.80 (thinking). Kind of unfortunate to see.
[1] https://github.com/Filimoa/rd-tablebench