Hardware Podcasts

AI Disruption: DeepSeek and Cerebras

Oxide and Friends · 6 Feb 2025 · 1h 31m · Episode 4

What this episode covers

Andy Hock and James Wang of Cerebras discuss how their wafer-scale silicon executes open-weight models like DeepSeek R1 Distill Llama 70B faster than any number of GPUs, achieving 57x speedup by keeping all 44GB of model weights in on-chip SRAM. They explain that the release of efficient, open models like DeepSeek and Llama transformed Cerebras from an exotic supercomputer vendor into a provider of commodity inference APIs, standardizing the unit of sale as tokens. The guests highlight that hardware-software codesign and the shift toward reasoning models allow smaller teams to compete with well-funded incumbents by optimizing algorithms for specific architectures rather than relying on brute-force scaling.

Most relevant to

companies scaling up

Topics

Mentioned

Companies
Cerebras, NVIDIA, OpenAI, Meta
Products
DeepSeek, Llama
Tools
PyTorch, CUDA

Related episodes from other shows

Other shows on the same ground, closest first.

  • The Circuit · 48m

    Episode 109: Unpacking the Nuances of Deep Seek with Austin Lyons

    Austin Lyons, founder of ChipStrat, joins Jay Goldberg to analyze Deep Seek, a self-funded AI lab from China that emerged from the quantitative hedge fund High Flyer. They discuss Deep Seek's technical innovations, including its efficient Mixture of Experts (MoE) architecture and the R1 reasoning model, which demonstrated that high-performance AI is possible despite US chip export controls limiting access to NVIDIA H100s. The conversation explores how Deep Seek's training efficiency and open-weight models have shifted the industry focus from pure pre-training scaling to inference optimization and post-training techniques.

    AI & machine learningChips, FPGAs & semiconductors
    7 chapters

  • The Circuit · 41m

    Episode 85: OpenAI leaves AI Silicon in the Dust

    Ben Bajarin and Jay Goldberg discuss how OpenAI's new model, which uses reinforcement learning and multi-level reasoning, drastically increases inference compute demands, leaving current silicon architectures behind. They analyze the competitive landscape, noting that while Nvidia dominates training, the inference market is becoming more open due to high costs and the need for edge deployment. The hosts also highlight Google's recent demos, including a podcast generator and a game engine, as evidence of rapid software innovation that continues to outpace hardware development.

    AI & machine learningChips, FPGAs & semiconductors
    7 chapters

  • The Circuit · 38m

    Episode 8: AI Infrastructure and Silicon Demands

    Ben Bajarin and Jay Goldberg discuss the immense compute demands of training large foundational models like GPT-3 and GPT-4, noting that Nvidia’s A100 and H100 GPUs dominate this $300 billion infrastructure market due to CUDA’s software moat. They analyze the bottleneck in inference scalability, arguing that while training is a small fraction of the market, inference costs will drop as incumbents like Google, Amazon, and Apple leverage their own silicon, leaving little room for new AI accelerator startups. The hosts conclude that generative AI represents a powerful new category of productivity software rather than a fundamental shift in human-computer interaction, driving data center Capex through natural upgrade cycles rather than complete rebuilds.

    Chips, FPGAs & semiconductorsConnected products & IoT
    8 chapters

  • The Circuit · 1h 02m

    EP 181: Cerebras Earnings, QCOM Investor Day, Micron Earnings and the Memory Mafia

    Ben Bajarin and Jay Goldberg analyze Cerebras's first public earnings, highlighting strong AI inference demand but investor concern over low gross margins caused by expensive wafer-scale packaging and capacity rental dynamics. They then dissect Qualcomm's Investor Day, focusing on its aggressive $15 billion data center revenue guidance, the 'Dragonfly' ARM CPU roadmap, and the strategic acquisition of Modular to address software integration challenges. Finally, the hosts examine Micron's massive earnings report, noting a 60% quarter-on-quarter memory price surge that surprised buyers like Apple, and discuss the 'Game of Thrones' style market dynamics and pricing leverage among Micron, SK Hynix, and Samsung.

    Pricing, margins & unit economicsChips, FPGAs & semiconductors
    6 chapters

  • The Circuit · 52m

    EP 161: TSMC Earnings, Cerebras S1, Custom Semi Rumors, Apple CEO Change

    Ben Bajarin and Jay Goldberg analyze TSMC's strong earnings and strategic ramp-up of N3 production to meet AI demand, while discussing the compute deficit impacting firms like Anthropic and Cerebras' S1 chip. They explore the fragmenting market for custom ASICs as Google considers design partners like Marvell and MediaTek, and weigh in on Tim Cook's departure from Apple, arguing his legacy was preserving the company's culture for John Ternus.

    Chips, FPGAs & semiconductorsAI & machine learning
    6 chapters

The summary, chapters and topics are generated from the episode’s audio by this directory, not written by Oxide and Friends. The episode itself belongs to its publisher.