Hardware Podcasts

Episode 36: The Wide World of Memory

The Circuit · 2 Oct 2023 · 37m · Episode 38

What this episode covers

Steven Woo, a Fellow at Rambus, explains the fundamental differences between memory and logic, detailing how DRAM innovation is measured by capacity and bandwidth rather than process nodes. He discusses the specific challenges of High Bandwidth Memory (HBM) for AI training, including stacking and power efficiency, and contrasts this with the cost constraints of standard DDR. The conversation covers the massive capital expenditures required for memory fabs, the role of chiplets in increasing package complexity, and the emerging competitive landscape involving Chinese manufacturer CXMT.

Most relevant to

large enterprises

Topics

Mentioned

Companies
Rambus, Micron, Samsung, Intel, AMD, Nvidia, CXMT
Products
HBM, DDR, LPDDR, GDDR, CXL

Related episodes from other shows

Other shows on the same ground, closest first.

  • EE Times Current · 40m

    AI and Semiconductor Memory: More, More, More

    Steven Woo, a Rambus fellow and distinguished inventor, explains how AI's exponential growth in model size and training data is creating unprecedented demands on memory capacity, bandwidth, and power efficiency. He details the industry's shift toward High Bandwidth Memory (HBM) and GDDR to handle these loads, while noting that emerging memories like ReRAM and MRAM face reliability hurdles. Woo also discusses how architectural innovations like Compute Express Link (CXL) and chip stacking are critical for reducing the power and latency associated with moving data between processors and memory.

    PCB design & layoutChips, FPGAs & semiconductors
    7 chapters

  • EE Times Current · 18m

    Designing the AI Factory from the Rack Up

    Neeraj Paliwal, Senior Vice President of Silicon IP Product Management at Synopsys, explains why the AI rack has replaced the data center as the fundamental unit of design and economics. He details the shift from 8kW to 130kW+ power densities in systems like NVIDIA's GB200 NVL72, driven by the need to optimize cost per token across distributed GPUs. Paliwal highlights that the industry is moving past single-die limits toward 3D stacking, custom HBM memory integration, and high-speed die-to-die interfaces like UCIe to overcome the memory and I/O bandwidth walls.

    Chips, FPGAs & semiconductorsBatteries & low-power design
    5 chapters

  • EE Times Current · 58m

    Fixing AI’s Bottlenecks: Memory, Scale, and Sparsity

    A panel of neuromorphic engineering experts, including Dr. Damien Querlioz, Dr. Julian Büchel, Professor Tamalika Banerjee, Dr. Maxence Ernoult, and Professor Steve Furber, discusses the bottlenecks in AI hardware at the Atoms to Bits conference. They debate the shift from scale-based emergence to engineered approaches, highlighting memory, connectivity, and sparsity as critical challenges. The discussion also covers the viability of emerging technologies, the role of public funding, and the potential for an AI market bubble.

    Chips, FPGAs & semiconductorsAI & machine learning
    9 chapters

  • EE Times Current · 7m

    TSMC Scraps a Batch of Wafers | AI Limitations | ISSCC

    Alan Patterson reports that TSMC scrapped a batch of 12 and 16 nanometer wafers at Fab 14B in Tainan due to substandard photo resist, cutting Q1 revenue forecasts by $550 million. Junko Yoshida highlights Noriko Arai's book 'AI versus Chutolin,' emphasizing that current AI lacks true understanding and cannot read textbooks. Dylan McGrath and Rick Merritt cover the 2019 ISSCC, noting Intel's production-ready embedded MRAM at the 22nm node and discussions on unsupervised learning.

    Chips, FPGAs & semiconductorsAI & machine learning
    5 chapters

  • EE Times Current · 47m

    Interview: Roboticist Ayanna Howard | Memories May Be Beautiful, And Yet | Math & Neptune

    Georgia Tech professor and roboticist Ayanna Howard discusses the intersection of AI and robotics, highlighting how humans overtrust robots in high-risk scenarios and the challenges of generalizing robotic behavior across different environments. EE Times contributor Gary Hilson analyzes the semiconductor memory market, explaining why emerging memories like MRAM and 3D XPoint struggle to displace established DRAM and Flash due to cost and manufacturing hurdles. The episode also previews the upcoming Boards and Solutions Conference, focusing on embedded computer boards for industrial automation, smart cities, and edge computing.

    Robotics & autonomyChips, FPGAs & semiconductors
    9 chapters

The summary, chapters and topics are generated from the episode’s audio by this directory, not written by The Circuit. The episode itself belongs to its publisher.