A recent analysis highlighted by MIT Technology Review puts the energy cost of generative AI into stark perspective. Generating a simple text response from Llama 3.1-405B—a model with 405 billion parameters, the adjustable “knobs” that enable prediction—requires on average 3,353 joules, nearly 1 watt-hour (Wh). Once cooling… Read More
Tag: gpgpu
Things to do in Denver when you’re 64-bit
When Apple announced last September their A7 chip had gone 64-bit, the congregation immediately swooned, but analysts reacted skeptically: “So what? Phones don’t need more memory, and there are no 64-bit apps.” Even pundits miss once in a while, and now the topic is how the chip industry is headed for 64-bit.… Read More
