Array
(
    [content] => 
    [params] => Array
        (
            [0] => /forum/index.php?threads/data-centre-emissions-are-soaring-%E2%80%93-it%E2%80%99s-ai-or-the-climate.21848/
        )

    [addOns] => Array
        (
            [DL6/MLTP] => 13
            [Hampel/TimeZoneDebug] => 1000070
            [SV/ChangePostDate] => 2010200
            [SemiWiki/Newsletter] => 1000010
            [SemiWiki/WPMenu] => 1000010
            [SemiWiki/XPressExtend] => 1000010
            [ThemeHouse/XLink] => 1000970
            [ThemeHouse/XPress] => 1010570
            [XF] => 2021770
            [XFI] => 1050270
        )

    [wordpress] => /var/www/html
)

Data centre emissions are soaring – it’s AI or the climate


1736760004921.png
 
Is the majority of "AI" energy use large farms doing training or is it mostly users of LLMs?
 
Is the majority of "AI" energy use large farms doing training or is it mostly users of LLMs?
An excellent question.

I'd read somewhere that a lot went on training. And in another place that one or more of the big companies did a 6 hourly internet scrape. Which tends to imply that training runs on a 6 hour cycle. But I really don't pay close attention, so could be getting some details wrong.

What does interest me is whether the training effort (in energy consumption terms) increases or declines over time going forwards. And what percentage of the total AI energy consumption this amounts to. On the one hand, the current AI model implies that you need to keep training forever as the data sets for most areas keep changing. On the other, presumably there's some law of diminishing returns in there somewhere and there ought to be a "good enough" state where you can stop. Or reduce the training frequency. Or partition the search space into stable and "needs more work" areas (perhasps they already do that).

I certainly think AI energy consumption is a first order problem right now.
 
Is the majority of "AI" energy use large farms doing training or is it mostly users of LLMs?

As highlighted in the following article, training involves both forward and backward passes, whereas inference requires only the forward pass. I believe inference might consume more energy due to smaller batch sizes and the large number of requests over the model's lifetime. Additionally, training is not a one-time process. For example, if the model is trained with data up to 2023, it would need to be retrained to incorporate information from 2024 and beyond, or to improve its overall performance.


With reasoning models, inference tends to consume significant energy, due to scaling test time compute.

 
Last edited:
. And in another place that one or more of the big companies did a 6 hourly internet scrape. Which tends to imply that training runs on a 6 hour cycle.
That is an interesting data point. On the flip side, I saw a tweet from Elon Musk that they train on pre-2024 or pre-2023 data because so much of the current content on the internet is AI generated. (Leading to poisoning of the source, so to speak). Of course this is referring to non-curated sources.

I don't know which trend is more likely.

What does interest me is whether the training effort (in energy consumption terms) increases or declines over time going forwards. And what percentage of the total AI energy consumption this amounts to. On the one hand, the current AI model implies that you need to keep training forever as the data sets for most areas keep changing. On the other, presumably there's some law of diminishing returns in there somewhere and there ought to be a "good enough" state where you can stop. Or reduce the training frequency. Or partition the search space into stable and "needs more work" areas (perhasps they already do that).

I certainly think AI energy consumption is a first order problem right now.
That is what I was thinking of when asking the question. A few other "power" modifiers:

Increase consumption over time:
- We are still discovering new usage models for AI
- Large AI efforts see size 'as a moat' to having a superior product
- As hardware for AI training becomes cheaper, more people will buy/use it
- Small customizable models running locally may become viable (= now adding 'small AI' to 'big AI')
- FOMO - more governments, businesses, and individuals are set to do more with AI

Decrease consumption over time:
- Small LLMs are making serious strides on performance
- Expanding on your "needs more work" comment: Fine tuning AI models post training does not appear to be a widespread practice (but it is being done). I suspect as more people become familiar with this,
- Specialized ASICs for training may be an opportunity to improve efficiency by orders of magnitude vs GPUs
- Government regulation (aka AI fears) limiting model sizes or capability in the future
- Memory technologies better suited for AI processing may scale better as demand increases. (i.e. HBM is better perf/watt than GDDR).
 
Back
Top