Who Is Groq? The Ex‑Google TPU Founder Behind the LPU

ByL

Groq is an AI‑chip company founded in 2016 by Jonathan Ross, the engineer who started Google’s Tensor Processing Unit (TPU) project as a 20% side project before it became core to Google’s AI stack. After leading early TPU development and working in the Google X “moonshot” group, Ross left to design a chip company around one idea: make AI inference fast, simple, and predictable.​

Groq’s main product is the Language Processing Unit (LPU), a custom accelerator built specifically for running large language models and other transformer workloads. Instead of optimizing for training, Groq tuned the whole stack—chip, on‑chip memory, compiler, and dataflow—for inferencing. The result is low latency and high tokens‑per‑second, which makes AI responses feel fast and responsive.​

Groq’s business has two layers. At the hardware level, it sells LPU‑based systems to customers that want their own on‑prem or data‑center infrastructure. On top of that, Groq runs GroqCloud, an API service that lets developers tap into LPU inference over the internet, similar to how they rent GPU instances from cloud providers today.

Chart comparing Nvidia, Groq and other AI accelerator vendors in the data center inference market, showing Groq’s LPU positioned as a competitive challenger.
While Nvidia is the dominant player in the AI accelerator market, Groq (with its LPU) is a key challenger in the AI inferencing space. (Source: Artificial Analysis)
Groq large language model leaderboard chart showing its LLM ranked among the most capable models by intelligence level, based on Artificial Analysis benchmarking.
Groq’s large language model (LLM) is also leading the pack in terms of its intelligence level (Source: Artificial Analysis)

​This post is for educational and informational purposes only, reflects personal opinions, and does not constitute investment or financial advice; please do your own research or consult a licensed advisor.

Similar Posts