Mozilla management was paid millions to develop a new “vision” of a theoretical future with AI chatbots
Is this llamafile?
The thing about LLMs is that no one knows how to write the ultra low level optimizations/runtimes, so they port others (llamafile largely borrows from llama.cpp AFAIK, albeit with some major contributions from their own devs).
Performance is insanely critical because they’re so darn hard to run, and new models/features come out weekly which no sane dev can keep up with without massive critical mass (like HF Transformers, mainly, with llama.cpp barely keeping up with some major jank).
So… I’m not sure what Mozilla is thinking here. They don’t have many of those kind of devs, they don’t have a GPU farm, they’re not even contributing to promising webassembly projects like mlc-llm. They’re just one of a bazillion companies that was ordered to get into AI with no real purpose or advantage. And while Gemma 2B may be the “first” model that’s kinda OK on average PCs, we’re still a long way away from easy mass local deployment.
Anyway, what I’m getting at is that I’m a local LLM tinkerer, and I’ve never touched or even looked at anything from Mozilla. The community would have if anything of theirs was super useful.
Deebster