According to a report by The Wall Street Journal, Nvidia is developing a new system tailored specifically for inference computing. Unlike training — where AI models learn from massive datasets — inference refers to the operational phase when systems like ChatGPT answer user queries, generate code, or interact with other software. As AI adoption expands across industries, inference has become one of the most commercially critical and hardware-intensive segments of the AI pipeline.
The new platform is expected to be introduced at Nvidia’s annual GTC developer conference in San Jose next month. Notably, the system will reportedly incorporate a chip designed by AI hardware startup Groq, signaling a strategic collaboration in a rapidly evolving competitive landscape.
Reuters said it could not independently verify the report. Neither Nvidia nor OpenAI immediately responded to requests for comment.
OpenAI’s Need for Faster Inference
Earlier this month, Reuters reported that OpenAI has been dissatisfied with the speed at which Nvidia’s existing hardware processes certain types of inference tasks — particularly those involving software development or AI-to-AI communication. As generative AI applications grow more sophisticated, the demand for near-instantaneous responses has intensified pressure on infrastructure providers.
According to one source cited by Reuters, OpenAI is seeking new hardware solutions that could eventually supply roughly 10% of its inference computing needs. The company has reportedly explored partnerships with emerging chipmakers, including Cerebras and Groq, in pursuit of faster inference capabilities.
However, Nvidia’s reported $20 billion licensing deal with Groq appears to have reshaped that landscape. One source told Reuters that the agreement effectively curtailed OpenAI’s direct talks with Groq, consolidating Nvidia’s position within the AI hardware supply chain.
Deepening Financial Ties
The reported chip development follows a deepening financial relationship between Nvidia and OpenAI. In September, Nvidia disclosed plans to invest as much as $100 billion into OpenAI as part of a broader arrangement that granted the chipmaker an equity stake in the startup. The deal also provided OpenAI with substantial capital to acquire advanced chips necessary to support its expanding AI services.
If unveiled as expected at GTC, Nvidia’s new inference platform could mark a significant step in addressing performance bottlenecks that have emerged as AI systems scale. With inference increasingly viewed as the next major battleground in AI infrastructure, the collaboration between Nvidia and Groq may signal a new phase in the race to power faster, more efficient generative AI systems worldwide.