Tons of work being done to improve the energy efficiency of ML models. We’re in the ENIAC days of AI right now. I’m not sure I see the problem other than that it would obviously be nicer if we could just build a time machine and steal an energy efficient AI from the year 2100? But in the real world, R&D takes time, and while, globally, we do need to reduce energy use, that doesn’t mean we should give up on R&D, especially when ML actually has the potential to help us achieve higher energy efficiencies across the entire economy.
Not like tons of uses for servers aren’t trivial and honestly kind of a waste. Okay, ML model’s energy use is a scandal, but Netflix and TikTok? Completely worth every joule.
While it is good to be cognizant of this, playing AAA games for the same amount of time as the inference (a few seconds ?) is the same as this, right? Since they use the same GPU on consumer hardware.
It’s probably a net savings over a digital artist creating images given the speed. Just powering your monitor for so much longer is going to take more power.
A high(er) end smartphone has a battery capacity of approx. 0.019kWh (5000mAh), a gtx3080 has a max power draw of 320W so running that (at max load) for two hours is 0.64kWh, which is equivalent to fully charging ~34 smartphones.
So the headline must be false, since you can generate a lot more than 34 generative AI images on a 3080 in 2 hours. That’s if you just include inference though.
I wonder if they are somehow trying to factor in the training costs.
The referenced part of the paper, for those interested in the maths.
Text-based tasks are, all things considered, more energy-efficient than image-based tasks, with image classification requiring less energy (median of 0.0068 kWh for 1,000 inferences) than image generation (1.35 kWh) and, conversely, text generation (0.042 KwH) requiring more than text classification (0.0023 kWh). For comparison, charging the average smartphone requires 0.012 kWh of energy 4, which means that the most efficient text generation model uses as much energy as 16% of a full smartphone charge for 1,000 inferences, whereas the least efficient image generation model uses as much energy as 950 smartphone charges (11.49 kWh), or nearly 1 charge per image generation, although there is also a large variation between image generation models, depending on the size of image that they generate.
No shit that using your PC for any purpose will consume electricity. A modern GPU can generate an image in a couple of seconds. Or I could just play a video game for an hour, and consume a few thousand times more energy
Yeah, I can’t imagine it’s that different from playing a demanding game. I hear my video card fans spin up harder and sustain that speed for the duration of a play session.
The model cards for Stable Diffusion 1.5 and 2.1 estimate the CO2 emissions as 11.25 tons and 12 tons for training. XL lacks the info.
A transatlantic flight (round-trip) is about 1 ton per pax. So, while every little bit helps, ML is not where you can make the big gains in lowering emissions.
Their work, which is yet to be peer reviewed, shows that while training massive AI models is incredibly energy intensive, it’s only one part of the puzzle.
For each of the tasks, such as text generation, Luccioni ran 1,000 prompts, and measured the energy used with a tool she developed called Code Carbon.
Generating 1,000 images with a powerful AI model, such as Stable Diffusion XL, is responsible for roughly as much carbon dioxide as driving the equivalent of 4.1 miles in an average gasoline-powered car.
AI startup Hugging Face has undertaken the tech sector’s first attempt to estimate the broader carbon footprint of a large language model.
The generative-AI boom has led big tech companies to integrate powerful AI models into many different products, from email to word processing.
Luccioni tested different versions of Hugging Face’s multilingual AI model BLOOM to see how many uses would be needed to overtake training costs.
The original article contains 1,021 words, the summary contains 153 words. Saved 85%. I’m a bot and I’m open source!