Hardware Podcasts

Designing the AI Factory from the Rack Up

EE Times Current · 21 Aug 2026 · 18m

What this episode covers

Neeraj Paliwal, Senior Vice President of Silicon IP Product Management at Synopsys, explains why the AI rack has replaced the data center as the fundamental unit of design and economics. He details the shift from 8kW to 130kW+ power densities in systems like NVIDIA's GB200 NVL72, driven by the need to optimize cost per token across distributed GPUs. Paliwal highlights that the industry is moving past single-die limits toward 3D stacking, custom HBM memory integration, and high-speed die-to-die interfaces like UCIe to overcome the memory and I/O bandwidth walls.

Most relevant to

large enterprises

Topics

Mentioned

Companies
Synopsys, NVIDIA
Products
GB200, NVL72, HBM
Also
UCIe

Related episodes from other shows

Other shows on the same ground, closest first.

  • The Circuit · 37m

    Episode 36: The Wide World of Memory

    Steven Woo, a Fellow at Rambus, explains the fundamental differences between memory and logic, detailing how DRAM innovation is measured by capacity and bandwidth rather than process nodes. He discusses the specific challenges of High Bandwidth Memory (HBM) for AI training, including stacking and power efficiency, and contrasts this with the cost constraints of standard DDR. The conversation covers the massive capital expenditures required for memory fabs, the role of chiplets in increasing package complexity, and the emerging competitive landscape involving Chinese manufacturer CXMT.

    Chips, FPGAs & semiconductorsManufacturing partners & offshoring
    9 chapters

  • The Circuit · 48m

    EP 180: HPE Reflections on AI Compute Racks, Analog Analog Analog Semis.

    Ben Bajarin and Jay Goldberg discuss findings from HPE Discover and Computex, focusing on how Dell and HPE are differentiating their AI infrastructure through custom rack engineering, hybrid cooling, and specialized networking strategies. They analyze the shift toward on-premises 'private AI factories' for enterprises and the growing importance of analog components and passive parts like MLCCs driven by the power demands of NVIDIA's Vera Rubin GPUs.

    Mechanical engineeringAnalog & power electronics
    9 chapters

  • The Circuit · 52m

    EP 163: Breaking the Memory Wall: Micron’s Strategy for the AI Era

    Jeremy Werner, SVP and GM of Micron’s Core Data Center Business Unit, explains how AI is shifting the memory industry from a cyclical model to sustainable growth driven by the 'memory wall' in inference. He details Micron's strategy to break this bottleneck through innovations like HBM4 and ultra-high-capacity SSDs, which address critical data center constraints of power and physical space. The conversation covers the technical hierarchy of memory, from HBM to context storage, and Micron's aggressive capacity expansion plans to meet surging demand for AI training and inference workloads.

    Chips, FPGAs & semiconductorsPCB design & layout
    7 chapters

  • Catalyst · 35m

    Inside Google’s massive AI capex (live)

    Amin Vahdat, Google’s chief technologist for AI infrastructure, explains how Google is shifting from training to inference, enabling smaller, distributed data centers and lowering reliability requirements to double compute capacity. He details Google's strategy of using behind-the-meter power as a bridge to grid interconnection and their milestone agreement with utilities for one gigawatt of demand response. Vahdat emphasizes that co-optimizing chip design, building cooling, and power generation creates superefficient 'AI factories,' while identifying power, chips, and construction as simultaneous rate limiters for growth.

    Energy & climate techChips, FPGAs & semiconductors
    8 chapters

  • The Circuit · 40m

    Episode 79: Not Your Grandfathers Data Center

    Ben Bajarin and Jay Goldberg discuss the recent volatility in the semiconductor stock market, attributing it to a 'bubble over' rather than fundamental shifts, while noting that AI's benefits are unevenly distributed. They analyze the evolving architecture of AI data centers, arguing that power constraints and liquid cooling requirements are forcing a shift from air-cooled racks to new designs with higher GPU-to-CPU ratios. The hosts also examine the automotive semiconductor sector, highlighting weak demand driven by high EV inventory in China and the divergence between analog and digital cyclicality.

    Chips, FPGAs & semiconductorsScaling production
    5 chapters

The summary, chapters and topics are generated from the episode’s audio by this directory, not written by EE Times Current. The episode itself belongs to its publisher.