Hardware Podcasts

Episode 70!!! NPUs!!!!!!!!

The Circuit · 2 Jun 2024 · 51m · Episode 71

What this episode covers

Ben Bajarin and Jay Goldberg host Austin Lyons and Paul Karazuba to discuss the architecture, market, and future of Neural Processing Units (NPUs). The group debunks the metric 'TOPS per watt' as meaningless, arguing that real-world performance depends on specific network workloads and latency. They explore how chipmakers like Qualcomm, Intel, and Apple are designing NPUs, often by licensing IP or evolving DSPs, and predict that NPUs will become ubiquitous in devices from smartphones to microcontrollers within three to four years.

Most relevant to

large enterprises

Topics

Mentioned

Companies
Expedera, Qualcomm, Intel, Apple, AMD, Movidius, Wavidea, NVIDIA
Products
Qualcomm Hexagon, Trainium, Inferentia, TPU, Copilot Plus PC, Snapdragon

Related episodes from other shows

Other shows on the same ground, closest first.

  • EE Times Current · 45m

    From Data Centers to Devices – The Ongoing [R]Evolution of AI Computing

    Matt Gutierrez, Senior Director of Marketing at Synopsys, discusses the migration of AI computing from centralized data centers to the network edge. He explains that this shift is driven by the need for low latency, data privacy, and power efficiency, enabled by smaller neural networks and specialized hardware like NPUs and TPUs. Gutierrez details how Synopsys supports this through its ARC processor IP portfolio and EDA tools, helping engineers manage the complexity of heterogeneous processing platforms for edge devices.

    AI & machine learningMicrocontrollers & dev platforms
    7 chapters

  • EE Times Current · 49m

    Can the Nvidia Monopoly on AI Chips Be Broken?

    Sally Ward-Foxton of EE Times discusses the AI chip market with hosts Dr. Sunny Bains and Dr. Giulia D’Angelo, analyzing the dominance of Nvidia and the potential for neuromorphic and specialized inference chips to break its monopoly. The conversation covers the spectrum of AI hardware from data center GPUs to ultra-low-power edge devices, highlighting the critical role of software stacks and latency in determining adoption. Professor Ralph Etienne-Cummings joins to critique the industry's focus on power versus learning capabilities and to discuss the future of neuromorphic computing, including memristors and photonic integration.

    Chips, FPGAs & semiconductorsAI & machine learning
    9 chapters

  • Embedded.fm · 1h 24m

    497: Everyone Likes Tiny

    Kwabena Agyeman, President and Co-Founder of OpenMV, discusses the launch of two new programmable camera systems, the OpenMV Cam N6 and the OpenMV AE3, via a Kickstarter campaign. He explains how integrating Neural Processing Units (NPUs) and Arm Helium SIMD instructions enables high-performance computer vision and edge AI on microcontrollers, achieving significant speedups while maintaining ultra-low power consumption. Agyeman details the software ecosystem, including MicroPython, TensorFlow Lite for Microcontrollers, and partnerships with Edge Impulse and Roboflow, which simplify model training and deployment for embedded developers.

    Embedded firmwareMicrocontrollers & dev platforms
    8 chapters

  • Moore's Lobby · 2m

    Get Ready For Season 4 of Moore's Lobby

    Host Daniel Bogdanoff introduces Season 4 of Moore's Lobby, previewing episodes with Jonathan Ross of Grok and Hugo Swartz of Qualcomm. Ross explains how Grok's chip architecture achieves peta-op performance by combining CPU speed with GPU throughput on older technology. Swartz discusses Qualcomm's evolution from modem and audio components to a comprehensive compute platform integrating CPU, GPU, DSPs, and camera processors for XR and metaverse applications.

    Chips, FPGAs & semiconductorsAI & machine learning
    4 chapters

  • EE Times Current · 34m

    Whither Semiconductors? The Wheels on the Bus Go ‘Round and ‘Round

    Jim McGregor, co-founder of Tirias Research, discusses the evolving competitive landscape of the semiconductor industry, identifying Intel, AMD, and NVIDIA (following its potential acquisition of Arm) as the new dominant trio. He explains how the convergence of computing, connectivity, and intelligence is blurring traditional boundaries, with companies like Qualcomm, Samsung, and MediaTek competing across smartphones, servers, and automotive sectors. McGregor highlights the shift toward heterogeneous computing, specialized accelerators like DPUs, and the move away from monolithic dies toward multi-chip modules and advanced packaging to overcome Moore's Law limitations.

    Chips, FPGAs & semiconductorsMechanical engineering
    8 chapters

The summary, chapters and topics are generated from the episode’s audio by this directory, not written by The Circuit. The episode itself belongs to its publisher.