Hardware Podcasts

Demystifying AI: How Neural Networks Like Transformers Really Work

EE Times Current · 27 Apr 2023 · 22m

What this episode covers

Gordon Cooper, Product Manager for AI and neural network processor IP at Synopsys, explains that generative AI models like ChatGPT are 'magic tricks' that identify word patterns in large datasets rather than understanding meaning. He details how Transformer architectures, specifically GPT-3 and GPT-4, utilize attention mechanisms to achieve high accuracy but require massive computational resources and parameters. Cooper highlights the challenge of deploying these models in embedded spaces, where power consumption and area constraints necessitate efficient algorithms and hybrid hardware-software solutions.

Most relevant to

large enterprises

Topics

Mentioned

Companies
Synopsys, TSMC, Samsung, Intel, Apple, NVIDIA, Samba Nova, TDK Venture, Google Brain
Products
ChatGPT, GPT-3, GPT-4, Novolith, AlexNet
Also
Recurrent Neural Networks, Convolutional Neural Networks

Related episodes from other shows

Other shows on the same ground, closest first.

  • The Circuit · 38m

    Episode 8: AI Infrastructure and Silicon Demands

    Ben Bajarin and Jay Goldberg discuss the immense compute demands of training large foundational models like GPT-3 and GPT-4, noting that Nvidia’s A100 and H100 GPUs dominate this $300 billion infrastructure market due to CUDA’s software moat. They analyze the bottleneck in inference scalability, arguing that while training is a small fraction of the market, inference costs will drop as incumbents like Google, Amazon, and Apple leverage their own silicon, leaving little room for new AI accelerator startups. The hosts conclude that generative AI represents a powerful new category of productivity software rather than a fundamental shift in human-computer interaction, driving data center Capex through natural upgrade cycles rather than complete rebuilds.

    Chips, FPGAs & semiconductorsConnected products & IoT
    8 chapters

  • Coredump Sessions · 55m

    #007: AI, Open Source, and the Future of Embedded Development: How Much Code Will We Actually Write?

    François Baldassari, CEO of Memfault, and Thomas Sarlandie, Field CTO of Memfault, host a panel with Jonathan Beri, CEO of Golioth, and Dan Mangum, CTO of Golioth, to discuss the intersection of AI, open source, and embedded development. The group analyzes the maturity of edge AI for audio and video processing, the challenges of integrating LLMs into hardware due to rapid model iteration, and the current state of using AI for firmware development. They conclude that while AI is transforming code generation and review, open source remains critical for training models and providing trusted, complex infrastructure like Zephyr and FreeRTOS.

    Embedded firmwareAI & machine learning
    9 chapters

  • The Circuit · 34m

    Episode 19: The On-Device AI Debate

    Ben Bajarin and Jay Goldberg discuss the economic and technical drivers shifting AI workloads from the cloud to on-device processing, highlighting companies like Qualcomm, Apple, and NVIDIA. They analyze the trade-offs between cloud-based model size and accuracy versus edge-based privacy and cost, noting that models like Llama and Stable Diffusion are being optimized for devices like the iPhone and Raspberry Pi. The hosts argue that while cloud AI offers superior capability, the high cost of inference and privacy concerns are pushing developers to adopt hybrid approaches and optimize silicon for power efficiency rather than raw throughput.

    Connected products & IoTMicrocontrollers & dev platforms
    10 chapters

  • OnTrack · 41m

    Edge Devices and LLMs: What's Ahead for AI

    Jan Jongboom, co-founder of Edge Impulse, discusses the feasibility of running Large Language Models (LLMs) on edge devices by reducing their size through knowledge distillation and synthetic data generation. He explains that while general-purpose MCUs can handle small, specialized models, RAM capacity is often the primary constraint rather than compute power. Jongboom highlights techniques like using GPT-4 to label data for training smaller, hyper-specific models that fit on microcontrollers like the Cortex-M4.

    AI & machine learningMicrocontrollers & dev platforms
    5 chapters

  • Oxide and Friends · 1h 31m

    AI Disruption: DeepSeek and Cerebras

    Andy Hock and James Wang of Cerebras discuss how their wafer-scale silicon executes open-weight models like DeepSeek R1 Distill Llama 70B faster than any number of GPUs, achieving 57x speedup by keeping all 44GB of model weights in on-chip SRAM. They explain that the release of efficient, open models like DeepSeek and Llama transformed Cerebras from an exotic supercomputer vendor into a provider of commodity inference APIs, standardizing the unit of sale as tokens. The guests highlight that hardware-software codesign and the shift toward reasoning models allow smaller teams to compete with well-funded incumbents by optimizing algorithms for specific architectures rather than relying on brute-force scaling.

    AI & machine learningChips, FPGAs & semiconductors
    10 chapters

The summary, chapters and topics are generated from the episode’s audio by this directory, not written by EE Times Current. The episode itself belongs to its publisher.