Hardware Podcasts

Episode 85: OpenAI leaves AI Silicon in the Dust

The Circuit · 16 Sept 2024 · 41m · Episode 85

What this episode covers

Ben Bajarin and Jay Goldberg discuss how OpenAI's new model, which uses reinforcement learning and multi-level reasoning, drastically increases inference compute demands, leaving current silicon architectures behind. They analyze the competitive landscape, noting that while Nvidia dominates training, the inference market is becoming more open due to high costs and the need for edge deployment. The hosts also highlight Google's recent demos, including a podcast generator and a game engine, as evidence of rapid software innovation that continues to outpace hardware development.

Most relevant to

large enterprises

Topics

Mentioned

Companies
OpenAI, Nvidia, Google, AMD, TSMC, Apple, Microsoft, Unity
Products
Grok, Blackwell, MI300X, Gemini, Google Illuminate, Google Game Engine
Tools
CUDA

Related episodes from other shows

Other shows on the same ground, closest first.

  • EE Times Current · 40m

    CEO Interview: Flex Logix’ Geoff Tate on AI | Open Source Hits Hardware

    Geoff Tate, CEO of Flex Logix, explains the distinction between AI training and inference, arguing that engineers must benchmark their specific neural network models on target chips rather than relying on generic metrics like TOPS, which he demonstrates can be misleading compared to real-world performance on models like YOLO V3. Industry analyst Kevin Krewell discusses the rise of open technologies in hardware, tracing the evolution from open source software like Linux to open RAN in networking and the growing ecosystem around the open-source RISC-V architecture as an alternative to proprietary instruction sets like ARM and x86.

    AI & machine learningChips, FPGAs & semiconductors
    6 chapters

  • Oxide and Friends · 1h 31m

    AI Disruption: DeepSeek and Cerebras

    Andy Hock and James Wang of Cerebras discuss how their wafer-scale silicon executes open-weight models like DeepSeek R1 Distill Llama 70B faster than any number of GPUs, achieving 57x speedup by keeping all 44GB of model weights in on-chip SRAM. They explain that the release of efficient, open models like DeepSeek and Llama transformed Cerebras from an exotic supercomputer vendor into a provider of commodity inference APIs, standardizing the unit of sale as tokens. The guests highlight that hardware-software codesign and the shift toward reasoning models allow smaller teams to compete with well-funded incumbents by optimizing algorithms for specific architectures rather than relying on brute-force scaling.

    AI & machine learningChips, FPGAs & semiconductors
    10 chapters

  • Built 2 Scale · 1h 11m

    Inside the emerging frontier of models, machines and manufacturing hell.

    Hosts Matt Perrott and Scott Wilcox discuss OpenAI's strategic pivot to hardware by hiring 40 Apple engineers, analyzing whether this signals a competition with Google's compute advantage or a setup for an Apple acquisition. They examine Meta's aggressive acquisition of Limitless Pendant and its privacy policy changes as evidence of a lack of clear vision compared to Elon Musk, while also covering Boom Supersonic's $300M pivot from jet turbines to natural gas energy for AI data centers. The episode further explores the impact of data center demand on construction wages, Travis Kalanick's Cloud Kitchens automating food production with point-solution robots, and the emerging Private Equity playbook of injecting AI logic into traditional businesses to 10x margins.

    Chips, FPGAs & semiconductorsMechanical engineering
    13 chapters

  • Built 2 Scale · 1h 08m

    Google TPU vs Nvidia GPUs: Token Per Watt Dominance, or the Era of Scaling Is Over?

    Founders Matt Perrott and Scott Wilcox discuss Google's Gemini 3.0 training on TPUs, which bypasses Nvidia's hardware to achieve a structural cost advantage, and the resulting threat to 'vibe coding' startups like Lovable. They analyze the 'age of scaling is over' consensus among researchers like Ilya Sutskever, arguing that future breakthroughs require real-world spatial intelligence rather than just more compute. The hosts also debate office culture, remote work dynamics, and the three critical roles founders must hire early: product, brand, and sales.

    Chips, FPGAs & semiconductorsAI & machine learning
    6 chapters

  • EE Times Current · 49m

    Can the Nvidia Monopoly on AI Chips Be Broken?

    Sally Ward-Foxton of EE Times discusses the AI chip market with hosts Dr. Sunny Bains and Dr. Giulia D’Angelo, analyzing the dominance of Nvidia and the potential for neuromorphic and specialized inference chips to break its monopoly. The conversation covers the spectrum of AI hardware from data center GPUs to ultra-low-power edge devices, highlighting the critical role of software stacks and latency in determining adoption. Professor Ralph Etienne-Cummings joins to critique the industry's focus on power versus learning capabilities and to discuss the future of neuromorphic computing, including memristors and photonic integration.

    Chips, FPGAs & semiconductorsAI & machine learning
    9 chapters

The summary, chapters and topics are generated from the episode’s audio by this directory, not written by The Circuit. The episode itself belongs to its publisher.