NVIDIA-Groq: The Occam's Razor Explanation

Maybe its not the 4D chess everyone thinks it is ...

Occam’s razor is a principle often attributed to 14th–century friar William of Ockham that says that if you have two competing ideas to explain the same phenomenon, you should prefer the simpler one.

What Happened?

Nvidia made its largest purchase ever in acquiring a “non-exclusive licensing agreement” of Groq’s assets for $20B. This has made early investors like Chamath Palihapitiya who invested $10M in the seed round for about a 20% stake in the company, suddenly even wealthier with an extra few billion over Christmas. You can read Chamath’s investment memo here.

Source: Chamath Palihapitiya on X.

The key distinction here is that this deal is not an acquisition of Groq, but more an acquihire of Groq’s best talent - the most important of which is Jonathan Ross, CEO of Groq. He founded Google’s TPU program in his 20% “pet project” time allocation that Google gives. In addition, Nvidia also have access to Groq hardware and IP through the licensing deal. There has been a lot of coverage on this online, so I won’t get into all the gnarly details. Dr. Ian Cutress has a nice post explaining the whole thing clearly.

To be clear, Groq’s architecture is very different from Google TPUs. At the most obvious level, TPUs use HBM memory, while Groq’s Language Processing Units (LPUs) use SRAM. Irrational Analysis has a fun writeup on Very Long Instruction Word (VLIW) architecture used by Groq LPUs in his usual inimitable style.

Here’s a short official announcement from Groq.

Conspiracy Theories Abound

Why Nvidia would buy Groq — a company almost written away as a failure — is the real question keeping the semiconductor corners of the internet abuzz with excitement.

Here are some online theories:

1. DRAM memory costs are so high that inferencing with SRAM is the only option.

2. Nvidia has realized that Google TPUs or Groq LPUs are the way, and their GPU approach has no future.

3. GPU for training + Groq LPU for inference.

4. Pay $20B now to prevent the rise of a new inference technology that could cost Nvidia $200B later.

5. Groq’s 1st generation chips were inferior, but Jensen saw something threatening in the 2nd and 3rd generation chips.

6. Jensen is planning Groq style SRAM-based inferencing chips for robotics.

7. Chamath told David Sacks to tell Donald Trump to tell Jensen Huang to buy Groq so that he could profit.

8. Grok raised capital from 1789 Capital which of which Donald Trump Jr is a partner, so Trump Jr could profit.

There are probably more and any of these have a non-zero chance of being true.

The Search For Occam’s Razor

In typical Occam’s Razor format, I have two competing theories derived from what I’ve heard so far. You can decide which one seems simpler, and therefore correct to you. Of course, both could be wrong. This is pure speculation in the absence of data.