Apple Develops Breakthrough Method for Running LLMs on iPhones

Lophostemon , 7 months ago

Why the hell do we want to encourage people running MLMS on our phones?!! I don’t want to be part of some stupid pyramid scheme nonsense.

flooppoolf , 7 months ago

Haha

LazaroFilm , 7 months ago

dōTERRA Phone

Tronn4 , 7 months ago

HerbaLife Galaxy S30

jsh , 7 months ago

😭

Tja , 7 months ago

Here, you dropped this: /s

Lophostemon , 7 months ago

/s is for weaklings.

HEAR ME ROAR.

4am , 7 months ago

Siri could suck an order of magnetize less and work offline, for starters

pineapplelover , 7 months ago

LLM≠MLM

Lophostemon , 7 months ago

youtu.be/8N_tupPBtWQ?si=bDNpaip3U5tDk0h5

fruitycoder , 7 months ago

Tbh I’m more excited to see someone do use webnn, webgpu and petals together. Building smaller tighter models is good too.

AceSLS , 7 months ago

This comment sums it up pretty nicely:

LOL innovative invention of swapping memory to storage…… maybe they can call it something cool like “cache”.

Apple being “innovative” my ass, lmao

cheese_greater , 7 months ago

Cache u inside

Hegar , 7 months ago

The easiest way to tell that something's not really innovative is if the person describing it uses the word innovative.

HeartyBeast , 7 months ago

Can you give an example of something that actually was innovative, that no-one called innovative?

Hegar , 7 months ago

The spoked wheel.

HeartyBeast , 7 months ago

Interestingly, it looks as if nothing was really called innovative before 1960, with usage peaking in 2000, and its now in decline

https://www.etymonline.com/word/innovative

Hegar , 7 months ago

Peaking in 2000 seems odd - I see or hear that word daily and that definitely wasn't the case back in 2000. Interesting!

GluWu , 7 months ago

Corndogs

lemmylommy , 7 months ago

Well, if that commenter had more than just a vague idea of caching and/or swapping, they would know that the right algorithm can make or break performance.

That paper is not “we invented caching”, but “this is how we make some certain models work well despite constraints imposed by RAM and flash storage.”

It’s a worthy job for an engineer or researcher. Not quite as innovative as the invention of the wheel, but still enough to write a paper on (and read it, if you can manage to understand it).

4am , 7 months ago

35 upvotes in the technology community…man you guys really are just all knee-jerk reactionaries and it really knowledgeable tech at all. git gud

guitarsarereal , 7 months ago (edited 7 months ago)

Everyone likes to trash machine learning because the power requirements are high, but what they don’t realize is that we’re in the very first days of this technology (well, first couple decades of the technology being around, first few years of it being advanced enough to have anything to show off). Every technology that got bundled together into your phone was equally as useless when it was first invented. Honestly, compared to the development of most other technologies I’ve looked at, the pace of development in AI has been shocking.

Literally once a week, I see some news story about AI researchers delivering an order of magnitude speedup in some aspect of AI inference. The technique described here apparently allows for a 20x speedup on GPU’s.

cybersandwich , 7 months ago

Whispercpp works off the ML cores on the m series chips. It’s faster than my 1080ti that I have in a server doing the same things–by orders of magnitude. And it sips power.

Purpose built chips can be super powerful for their specific purposes.

cheese_greater , 7 months ago

Make Siri Great For Once

LazaroFilm , 7 months ago

Huhum…?

Still working on that…

I’m sorry, try again later.

cheese_greater , 7 months ago

You’re triggering me lol

coolmojo , 7 months ago

Found the following websites about you’re triggering me lol.

cheese_greater , 7 months ago

Sigh! [unzips] Go ahead…