Hardware Podcasts

Episode 75: The AI Big $$$ Question

The Circuit · 16 Jul 2024 · 41m · Episode 76

What this episode covers

Ben Bajarin and Jay Goldberg discuss the skepticism surrounding the monetization of the massive $600 billion investment in AI infrastructure, citing reports from Sequoia and Goldman Sachs. They analyze why current AI applications, such as Microsoft Copilot and coding assistants, offer only modest productivity gains that do not justify the capital expenditure. The hosts also examine the critical constraints of power generation, noting the need for 47 gigawatts of new capacity by 2030, and compare the current AI build-out to historical infrastructure bubbles like railroads and fiber optics.

Most relevant to

large enterprises

Topics

Mentioned

Companies
Sequoia, Goldman Sachs, Microsoft, Amazon, Apple, Google, Nvidia, Mobvoi, OpenAI
Products
Apple Intelligence, Microsoft Copilot, ChatGPT

Related episodes from other shows

Other shows on the same ground, closest first.

  • Catalyst · 54m

    A skeptic’s take on AI electricity load growth

    Jonathan Koomey, founder of Koomey Analytics, challenges the prevailing narrative of explosive AI-driven electricity load growth, arguing that historical data shows efficiency gains typically outpace demand increases. He explains that current forecasts often rely on Nvidia’s sales projections rather than actual service demand, ignoring the potential for significant efficiency improvements in software, algorithms, and specialized hardware. Koomey suggests that infrastructure constraints, such as transformer and labor shortages, will force the industry to prioritize efficiency, leading to a growth pattern similar to the 2010-2018 period where compute output surged while electricity use remained modest.

    Energy & climate techAI & machine learning
    11 chapters

  • Catalyst · 35m

    Inside Google’s massive AI capex (live)

    Amin Vahdat, Google’s chief technologist for AI infrastructure, explains how Google is shifting from training to inference, enabling smaller, distributed data centers and lowering reliability requirements to double compute capacity. He details Google's strategy of using behind-the-meter power as a bridge to grid interconnection and their milestone agreement with utilities for one gigawatt of demand response. Vahdat emphasizes that co-optimizing chip design, building cooling, and power generation creates superefficient 'AI factories,' while identifying power, chips, and construction as simultaneous rate limiters for growth.

    Energy & climate techChips, FPGAs & semiconductors
    8 chapters

  • The Manufacturing Executive · 32m

    Surging AI and Data Center CapEx: Are We on the Verge of a Bubble? w/ Wasif Khan

    Wasif Khan, CFO and Co-Founder of viaPhoton, argues that the surge in AI data center capital expenditure is not a bubble but a transformative shift comparable to the move to cloud computing, driven by massive potential labor efficiency gains. He advises manufacturing leaders to navigate this volatile market by keeping fixed costs low, maintaining flexible balance sheets with broad loan covenants, and focusing on team morale. Khan also discusses viaPhoton's culture of giving young engineers significant responsibility to foster growth in a high-stakes environment.

    Growth, exits & acquisitionsLeadership, teams & culture
    6 chapters

  • Built 2 Scale · 14m

    AI NEWS | Episode 32

    Hosts Matt Perrott and Scott Wilcox discuss OpenAI's strategic pivot to hardware, highlighted by hiring 40 Apple engineers and the vision of AI models running on network nodes without traditional operating systems. They analyze Meta's aggressive acquisition of Limitless AI wearables, noting the privacy policy changes and the company's pivot away from open source AI. The episode also covers Boom Supersonic's pivot to energy infrastructure for data centers, space-based data centers proposed by Gavin Baker, and the resulting 25-30% surge in construction wages due to AI infrastructure demand.

    Mechanical engineeringEnergy & climate tech
    6 chapters

  • The Amp Hour · 1h 05m

    All Heat, No Useful Work

    Dave Jones and Chris Gammell discuss the massive energy demands and infrastructure challenges of AI data centers, noting that 100% of input power becomes heat and requiring extensive cooling and diesel backup. They cover supply chain constraints, including a three-year backlog for transformers and a 40% surge in bareboard PCB prices driven by AI server demand. The conversation shifts to practical electronics work, where Chris details building a power-injecting Tag-Connect debug board and using Claude to implement SMP firmware updates over web serial, while Dave shares his frustration with a solar monitoring service's price hike and his son's struggles with Linux at school.

    PCB design & layoutEmbedded firmware
    7 chapters

The summary, chapters and topics are generated from the episode’s audio by this directory, not written by The Circuit. The episode itself belongs to its publisher.