Meta: Our next-generation Meta Training and Inference Accelerator

[Daniel Nenni]

Takeaways​

• We’re sharing details about the next generation of the Meta Training and Inference Accelerator (MTIA), our family of custom-made chips designed for Meta’s AI workloads.
• 
  
• This latest version shows significant performance improvements over MTIA v1 and helps power our ranking and recommendation ads models.
• MTIA is part of our growing investment in our AI infrastructure and will complement our existing and future AI infrastructure to deliver new and better experiences across our products and services.

The next generation of Meta’s large-scale infrastructure is being built with AI in mind, including supporting new generative AI (GenAI) products and services, recommendation systems, and advanced AI research. It’s an investment we expect will grow in the years ahead as the compute requirements to support AI models increase alongside the models’ sophistication.

Last year, we unveiled the Meta Training and Inference Accelerator (MTIA) v1, our first-generation AI inference accelerator that we designed in-house with Meta’s AI workloads in mind – specifically our deep learning recommendation models that are improving a variety of experiences across our products.

MTIA is a long-term venture to provide the most efficient architecture for Meta’s unique workloads. As AI workloads become increasingly important to our products and services, this efficiency will improve our ability to provide the best experiences for our users around the world. MTIA v1 was an important step in improving the compute efficiency of our infrastructure and better supporting our software developers as they build AI models that will facilitate new and better user experiences.

Now, we’re sharing details about the next generation of MTIA.......

Our next generation Meta Training and Inference Accelerator

We are sharing details of our next generation chip in our Meta Training and Inference Accelerator (MTIA) family. MTIA is a long-term bet to provide the most efficient architecture for Meta’s unique workloads.

ai.meta.com

 

[Mooredaddy]

TSMC N5. Thanks for telling us META, unlike ALPHABET

 

[Daniel Nenni]

TSMC N5. Thanks for telling us META, unlike ALPHABET

Alphabet is using TSMC N5 as well. Alphabet, Meta, and Amazon are working on TSMC N3 now. I just don't see how Groq and the others are going to compete with the big cloud companies who make their own Silicon.

 

[milesgehm]

nice article. I was suprised that the voltage went up.

 

[Daniel Nenni]

I was surprised at the huge TPD increase. Is that really a good thing? That's a lot of heat.

 

[Mooredaddy]

I was surprised at the huge TPD increase. Is that really a good thing? That's a lot of heat.

That leaped out to me too. Smaller process + huge TPD increase. Brute force technique

 

[hist78]

Alphabet is using TSMC N5 as well. Alphabet, Meta, and Amazon are working on TSMC N3 now. I just don't see how Groq and the others are going to compete with the big cloud companies who make their own Silicon.

Do you think Intel Gaudi 3 is in an awkward situation too?

 

[Xebec]

Power is just really another “knob”; they may have decided that they needed the throughput more and were OK with the power budget this time.

Voltage going up with a frequency rise of almost 70% seems reasonable.