The real advances we will see in the coming years will be postprocessing. We don’t need new LLMs, we need new ways to make use of the data we already have models for.
The top entry there – has, as accelerator cards, 36,992 MI250Xs, which are apparently 500W each.
It’s the highest compute card count other than one using Intel Max 1550s, which has 63,744 cards, which are 600W each.
I’m not sure that the TOP500 is necessarily the best ranking here, since I don’t know if it’s the best metric for GPU-style compute, and the CPUs are also a factor. But at least going off what he’s saying, it’d probably rank above those for GPU power consumption (and compute capacity).
Could you please not disparage an entire language and ethnic group? There’s an entire grocery list of things you could insult Musk about, even while including Putin, to not bring the Russian language into it.
After “you have some people speaking Russian there” became a reason for invading a country, the entire language and ethnic group is indeed viewed less favorably.