OpenAI partners with Broadcom to develop custom AI inference chip

Move signals strategic pivot for OpenAI which relies on Nvidia's GPUs for both training, operational capabilities

OpenAI is collaborating with Broadcom Inc. to create a specialized artificial intelligence chip focused on enhancing the efficiency of AI model inference, which is the application of trained models to real-world data.

This move signals a strategic pivot for OpenAI, which has heavily relied on Nvidia's graphics processing units (GPUs) for both training and operational capabilities.

The partnership also involves Taiwan Semiconductor Manufacturing Company (TSMC), the world’s largest contract chip manufacturer, known for its expertise in producing high-performance chips.

Although discussions are still in their early phases, sources indicate that OpenAI has been investigating this custom chip design for about a year.

The goal is to develop chips optimized for running AI models after they have been trained, addressing the growing demand for efficient AI processing.

OpenAI's shift in strategy comes as the AI landscape evolves, with a significant surge in the need for computing power to support increasingly complex AI applications.

Traditionally, the market has been dominated by Nvidia, which commands over 80% of the share in AI training chips.

However, OpenAI's engagement with Broadcom is part of a broader industry trend to diversify chip supply chains in response to the escalating demand for AI technologies.

In a notable change from its previous ambitions, OpenAI is scaling back its plans to establish its own chip manufacturing facilities, or foundries, due to the considerable time and capital investment required.

Instead, the company is now focusing on collaborations with established partners to expedite the production of its custom chips.

This approach mirrors strategies adopted by larger tech companies like Amazon, Meta, and Microsoft, who are also exploring alternative chip suppliers to mitigate risks associated with Nvidia's dominance.

The collaboration with Broadcom has already had a positive impact on the latter's stock, which rose by 4.2% following the news.

Broadcom specializes in application-specific integrated circuits (ASICs) and has a diverse clientele, including major players like Google and Meta, highlighting its capability in chip design and production.

Analysts predict that the need for inference chips—essential for deploying AI models—will soon surpass that for training chips as more companies integrate AI into their operations.

OpenAI's planned custom chip is expected to enter production by 2026, although this timeline may change based on various factors.

OpenAI's financial considerations also play a critical role in this strategy.

The company is projected to incur a $5 billion loss this year, despite generating approximately $3.7 billion in revenue.

The high costs associated with AI infrastructure, including hardware, cloud services, and electricity, represent significant operational challenges.

To address these issues, OpenAI is exploring partnerships and investments aimed at strengthening its data center capabilities, which are vital for supporting the anticipated growth in AI applications.

Additionally, OpenAI is diversifying its chip sourcing strategy by incorporating AMD chips alongside Nvidia's offerings.

AMD's recent introduction of the MI300X chip is part of the company's effort to capture a portion of the AI chip market, projected to be worth billions.

As OpenAI advances its partnership with Broadcom, the implications for the broader AI sector could be profound, potentially reshaping how companies approach AI deployment and the infrastructure needed to support it.

The collaboration underscores the critical importance of specialized hardware in the rapidly evolving field of artificial intelligence, positioning OpenAI to better meet the increasing demands of its services.