Semiwiki on LinkedIn Semiwiki on Facebook Semiwiki on Twitter Register / Log in 
Array
(
    [content] => 
    [params] => Array
        (
            [0] => /forum/index.php?threads/metas-first-bespoke-ai-chips.17994/
        )

    [addOns] => Array
        (
            [DL6/MLTP] => 13
            [Hampel/TimeZoneDebug] => 1000070
            [SV/ChangePostDate] => 2010200
            [SemiWiki/Newsletter] => 1000010
            [SemiWiki/WPMenu] => 1000010
            [SemiWiki/XPressExtend] => 1000010
            [ThemeHouse/XLink] => 1000970
            [ThemeHouse/XPress] => 1010570
            [XF] => 2021370
            [XFI] => 1050270
        )

    [wordpress] => /var/www/html
)

Meta's first bespoke AI chips

B

blueone

Well-known member
www.zdnet.com

Meta unveils its first custom AI chip

The circuits come with software optimized to run PyTorch, and emphasize the task of making recommendations.
www.zdnet.com www.zdnet.com

Definitely going for efficiency over highest processing power, which IMO is smarter for initial custom hardware designs.

7nm TSMC.

64 custom PEs for acceleration, lots of memory channels for bandwidth, 128MB of on-die SRAM.

A couple of RISC-V cores, not Arm.
 
More technical details can be found here:

ai.facebook.com

MTIA v1: Meta’s first-generation AI inference accelerator

In 2020, we initiated the Meta Training and Inference Accelerator (MTIA) family of chips to support our evolving AI workloads, starting with an inference accelerator ASIC for deep learning recommendation models (DLRMs).
ai.facebook.com ai.facebook.com

"Lessons for the future

Building custom silicon solutions, especially for the first time, is a significant undertaking. From this initial program, we have learned invaluable lessons that we are incorporating into our roadmap, including architectural insights and software stack enhancements that will lead to improved performance and scale of future systems.

The challenges we need to address are becoming increasingly complicated. Looking at historical trends in the industry for scaling compute, as well as memory and interconnect bandwidth, we can see that memory and interconnect bandwidth are scaling at a much lower pace compared with compute over the last several generations of hardware platforms.

The lagging performance of memory and interconnect bandwidth has also manifested itself in the final performance of our workloads as well. For example, we see a significant portion of a workload’s execution time spent on networking and communication.

Moving forward, as part of building a better and more efficient solution, we are focused on striking a balance between these three axes (compute power, memory bandwidth, and interconnect bandwidth) to achieve the best performance for Meta’s workloads. This is an exciting journey, and we’re just getting started."
 
hist78 said:
More technical details can be found here:

ai.facebook.com

MTIA v1: Meta’s first-generation AI inference accelerator

In 2020, we initiated the Meta Training and Inference Accelerator (MTIA) family of chips to support our evolving AI workloads, starting with an inference accelerator ASIC for deep learning recommendation models (DLRMs).
ai.facebook.com ai.facebook.com

"Lessons for the future

Building custom silicon solutions, especially for the first time, is a significant undertaking. From this initial program, we have learned invaluable lessons that we are incorporating into our roadmap, including architectural insights and software stack enhancements that will lead to improved performance and scale of future systems.

The challenges we need to address are becoming increasingly complicated. Looking at historical trends in the industry for scaling compute, as well as memory and interconnect bandwidth, we can see that memory and interconnect bandwidth are scaling at a much lower pace compared with compute over the last several generations of hardware platforms.

The lagging performance of memory and interconnect bandwidth has also manifested itself in the final performance of our workloads as well. For example, we see a significant portion of a workload’s execution time spent on networking and communication.

Moving forward, as part of building a better and more efficient solution, we are focused on striking a balance between these three axes (compute power, memory bandwidth, and interconnect bandwidth) to achieve the best performance for Meta’s workloads. This is an exciting journey, and we’re just getting started."
Click to expand...
A great find. Thanks.
 
Daniel Nenni said:
Meta, Google, Microsoft and Amazon are all at TSMC. N5 in progress and N3 coming up. Fast times at TSMC.

ai.facebook.com

MTIA v1: Meta’s first-generation AI inference accelerator

In 2020, we initiated the Meta Training and Inference Accelerator (MTIA) family of chips to support our evolving AI workloads, starting with an inference accelerator ASIC for deep learning recommendation models (DLRMs).
ai.facebook.com ai.facebook.com
Click to expand...

You can add Ampere Computing/Oracle at TSMC to the list.

This growing list of big companies with in-house chip products is a serious problem for Intel now and in the coming years.
 
hist78 said:
You can add Ampere Computing/Oracle at TSMC to the list.

This growing list of big companies with in-house chip products is a serious problem for Intel now and in the coming years.
Click to expand...
A while ago I predicted Ampere would get acquired by a cloud company, but I was figuring it would be Microsoft for Azure. I know Oracle is an Ampere investor, but Oracle's cloud business probably isn't big enough to support Ampere. And a lot of Oracle's cloud is still running on SPARC.

AMD is at risk too, but probably not as much as Intel.
 
Barnsley said:
What does the chip do?

Get folk ads faster on their feed?
Click to expand...

You can read about that in the cited article:

"Meta describes the chip as being tuned for one particular type of AI program: deep learning recommendation models. These are programs that can look at a pattern of activity, such as clicking on posts on a social network, and predict related, possibly relevant material to recommend to the user. "
 
Maxim said:
You can read about that in the cited article:

"Meta describes the chip as being tuned for one particular type of AI program: deep learning recommendation models. These are programs that can look at a pattern of activity, such as clicking on posts on a social network, and predict related, possibly relevant material to recommend to the user. "
Click to expand...

This functionality got anything useful to do?
 
hist78 said:
This is an arms race.

www.theregister.com

Microsoft seeks electrical engineers for custom DC chips

Redmond see, Redmond do... what AWS and Google are also doing
www.theregister.com www.theregister.com
Click to expand...
Agreed. And Microsoft is behind Amazon, Google, and Meta in chip development. I think they know it and are desperate, otherwise they would not have acquired Fungible, which had the least successful of the startup DPU projects in the industry. Several people I know think they acquired Fungible just for the chip design team. But as one would expect, that wasn't enough. I still can't believe AMD paid $1.9B for Pensando, which was better than Fungible, but the value is difficult to reconcile with the price. Nvidia still has far and away the best DPU with Bluefield 3. Although the proprietary nature of all of these DPUs makes me wonder if there is a sustainable market for them.
 
blueone said:
Agreed. And Microsoft is behind Amazon, Google, and Meta in chip development. I think they know it and are desperate, otherwise they would not have acquired Fungible, which had the least successful of the startup DPU projects in the industry. Several people I know think they acquired Fungible just for the chip design team. But as one would expect, that wasn't enough. I still can't believe AMD paid $1.9B for Pensando, which was better than Fungible, but the value is difficult to reconcile with the price. Nvidia still has far and away the best DPU with Bluefield 3. Although the proprietary nature of all of these DPUs makes me wonder if there is a sustainable market for them.
Click to expand...

Microsoft has a very large chip team so I would not count them out. The difference I see is that companies like Amazon and Google are spending large sums for chip design while others are following the fabless model of cutting budgets/corners to protect chip margins.
 
You must log in or register to reply here.
Back
Top