Hardware Podcasts

Can chip efficiency slow AI's energy demand?

Catalyst · 18 Jul 2024 · 44m

What this episode covers

Christian Belady, former Microsoft VP and creator of the PUE metric, explains that while chip efficiency gains from NVIDIA's Blackwell and Google's Trillium TPUs are significant, Jevons Paradox ensures total energy demand will rise as compute becomes cheaper and more capable. He argues that power availability, alongside labor and supply chain constraints, is now the primary bottleneck for AI growth, making it impossible to simply plug in gigawatt-scale data centers. Belady advocates for integrated demand response models, citing a 2016 Microsoft project at Cheyenne, and calls for deeper collaboration between hyperscalers and utilities to manage grid constraints. Finally, he introduces the concept of 'nature-positive' data centers, which aim to improve local ecosystem health rather than just minimizing carbon emissions.

Most relevant to

large enterprises

Topics

Mentioned

Companies
NVIDIA, Google, Microsoft
Products
Blackwell, Trillium TPUs
Also
PUE, Moore's Law, Cheyenne

Related episodes from other shows

Other shows on the same ground, closest first.

  • The Amp Hour · 1h 05m

    All Heat, No Useful Work

    Dave Jones and Chris Gammell discuss the massive energy demands and infrastructure challenges of AI data centers, noting that 100% of input power becomes heat and requiring extensive cooling and diesel backup. They cover supply chain constraints, including a three-year backlog for transformers and a 40% surge in bareboard PCB prices driven by AI server demand. The conversation shifts to practical electronics work, where Chris details building a power-injecting Tag-Connect debug board and using Claude to implement SMP firmware updates over web serial, while Dave shares his frustration with a solar monitoring service's price hike and his son's struggles with Linux at school.

    PCB design & layoutEmbedded firmware
    7 chapters

  • The Circuit · 1h 04m

    EP 186: Mobile Bottlenecks, Skyrocketing AI CapEx, and the Compute Crunch

    Hosts Ben Bajarin and Jay Goldberg analyze recent earnings from Apple, Qualcomm, ARM, and MediaTek, highlighting severe mobile supply chain bottlenecks and soft demand driven by elevated memory costs and foundry capacity constraints. They discuss skyrocketing hyperscaler CapEx from Amazon, Microsoft, and Meta, focusing on the transition from AI training to inference, the rise of behind-the-meter power solutions like Bloom Energy, and critical labor shortages in electrical trades. The episode concludes with an examination of market volatility linked to hedge fund unwinds and the increasing reliance on debt financing for data center expansion.

    Sourcing & supply chainEnergy & climate tech
    10 chapters

  • The Circuit · 43m

    Ep 160: More good News for Intel, AI Model Trends, Future of Datacenter

    Ben Bajarin and Jay Goldberg analyze Intel's recent strategic moves, including a partnership with Tesla for its 'Terafab' project and co-design work with Google on its IPU, interpreting these as validation for Intel's advanced packaging and foundry capabilities. They discuss the broader semiconductor market's valuation disconnect, noting how investors are shifting focus from established compute giants to obscure optical and mineral supply chain bottlenecks. The hosts also examine the future of data centers, arguing that extreme co-optimization of power (800 VDC) and cooling is becoming essential to unlock compute capacity, and cover Anthropic's rapid product rollout, including its Word connector and 'Mythos' model, while highlighting rising token costs and capacity constraints.

    Chips, FPGAs & semiconductorsEnergy & climate tech
    6 chapters

  • EE Times Current · 18m

    Designing the AI Factory from the Rack Up

    Neeraj Paliwal, Senior Vice President of Silicon IP Product Management at Synopsys, explains why the AI rack has replaced the data center as the fundamental unit of design and economics. He details the shift from 8kW to 130kW+ power densities in systems like NVIDIA's GB200 NVL72, driven by the need to optimize cost per token across distributed GPUs. Paliwal highlights that the industry is moving past single-die limits toward 3D stacking, custom HBM memory integration, and high-speed die-to-die interfaces like UCIe to overcome the memory and I/O bandwidth walls.

    Chips, FPGAs & semiconductorsBatteries & low-power design
    5 chapters

  • Moore's Lobby · 56m

    Aligning Passion With Purpose: A Journey to Electrification and Sustainability

    Dr. Mital Kanabar, Senior Director of Innovation at GE Vernova, discusses the 'energy trilemma' of security, affordability, and sustainability as power demand surges from EVs and AI data centers. He explains how GE Vernova's GridBeats platform uses software-defined automation and edge AI to manage bidirectional power flow and grid stability without massive hardware upgrades. The conversation covers the role of small modular reactors (SMRs), cybersecurity in operational technology, and the workforce challenges facing the utility industry.

    Energy & climate techAI & machine learning
    10 chapters

The summary, chapters and topics are generated from the episode’s audio by this directory, not written by Catalyst. The episode itself belongs to its publisher.