# Episode 19: The On-Device AI Debate

- Show: [The Circuit](https://hardwarepodcasts.com/podcast/the-circuit)
- Published: 2023-05-22
- Length: 34m
- Episode: 20
- Page: https://hardwarepodcasts.com/podcast/the-circuit/2023-05-22-episode-19-the-on-device-ai-debate
- Listen: https://share.transistor.fm/s/7cded877

## Summary

Ben Bajarin and Jay Goldberg discuss the economic and technical drivers shifting AI workloads from the cloud to on-device processing, highlighting companies like Qualcomm, Apple, and NVIDIA. They analyze the trade-offs between cloud-based model size and accuracy versus edge-based privacy and cost, noting that models like Llama and Stable Diffusion are being optimized for devices like the iPhone and Raspberry Pi. The hosts argue that while cloud AI offers superior capability, the high cost of inference and privacy concerns are pushing developers to adopt hybrid approaches and optimize silicon for power efficiency rather than raw throughput.

## Chapters

- [0:17](https://hardwarepodcasts.com/podcast/the-circuit/2023-05-22-episode-19-the-on-device-ai-debate#t=17) Introduction: The Cloud vs. On-Device AI Consensus
- [2:05](https://hardwarepodcasts.com/podcast/the-circuit/2023-05-22-episode-19-the-on-device-ai-debate#t=125) The Shift to Edge AI: Google Memo and Open Source Community
- [4:36](https://hardwarepodcasts.com/podcast/the-circuit/2023-05-22-episode-19-the-on-device-ai-debate#t=276) The Economic Paradox: Cloud Accessibility vs. Device Compute Limits
- [6:04](https://hardwarepodcasts.com/podcast/the-circuit/2023-05-22-episode-19-the-on-device-ai-debate#t=364) Technical and Economic Barriers: Memory, Cost, and Autonomy
- [8:29](https://hardwarepodcasts.com/podcast/the-circuit/2023-05-22-episode-19-the-on-device-ai-debate#t=509) Semiconductor Advocacy: Qualcomm, Apple, and the Expectations Gap
- [10:45](https://hardwarepodcasts.com/podcast/the-circuit/2023-05-22-episode-19-the-on-device-ai-debate#t=645) AI Hype Cycle and Realistic Use Cases: Coding and Writing
- [12:19](https://hardwarepodcasts.com/podcast/the-circuit/2023-05-22-episode-19-the-on-device-ai-debate#t=739) Privacy, Accuracy, and Hybrid Workflows: Notion and ChatGPT
- [14:33](https://hardwarepodcasts.com/podcast/the-circuit/2023-05-22-episode-19-the-on-device-ai-debate#t=873) Economics of Compute and Qualcomm's Power-Optimized Silicon
- [17:32](https://hardwarepodcasts.com/podcast/the-circuit/2023-05-22-episode-19-the-on-device-ai-debate#t=1052) Redefining Performance Metrics: Tops, Inference per Watt, and Human Intervention
- [25:55](https://hardwarepodcasts.com/podcast/the-circuit/2023-05-22-episode-19-the-on-device-ai-debate#t=1555) Measuring AI Gains: Productivity, Clock Cycles, and Future Standards

## Topics

- [Connected products & IoT](https://hardwarepodcasts.com/topic/connected-products-and-iot)
- [Microcontrollers & dev platforms](https://hardwarepodcasts.com/topic/microcontrollers-and-platforms)
- [Pricing, margins & unit economics](https://hardwarepodcasts.com/topic/pricing-and-unit-economics)
- [Security & privacy](https://hardwarepodcasts.com/topic/security-and-privacy)
- [Chips, FPGAs & semiconductors](https://hardwarepodcasts.com/topic/chips-and-fpgas)

## Most relevant to

large enterprises

## Mentioned

- Qualcomm (company)
- Apple (company)
- NVIDIA (company)
- Open AI (company)
- Google (company)
- ChatGPT (product)
- Llama (product)
- Stable Diffusion (product)
- Raspberry Pi (product)
- Notion (product)
- Jasper (product)
- GPT (product)
- Bard (product)

## Related episodes from other shows

- [235. Arm’s Vince Jesaitis on How AI Is Moving from the Cloud to Your Pocket](https://hardwarepodcasts.com/podcast/the-next-byte-podcast/2025-12-09-235-arms-vince-jesaitis-on-how-ai-is-moving-from-the-cloud-to-your-pocket) (The Next Byte Podcast)
- [145. The Prius of Artificial Intelligence: Hybrid AI Models](https://hardwarepodcasts.com/podcast/the-next-byte-podcast/2023-10-24-145-the-prius-of-artificial-intelligence-hybrid-ai-models) (The Next Byte Podcast)
- [Chips at the Edge: Innovation for AI at Edge and Automotive](https://hardwarepodcasts.com/podcast/ee-times-current/2026-01-14-chips-at-the-edge-innovation-for-ai-at-edge-and-automotive) (EE Times Current)
- [AI at the Edge: Justin Schneck on the Future of Embedded Devices](https://hardwarepodcasts.com/podcast/over-the-air/2025-05-29-ai-at-the-edge-justin-schneck-on-the-future-of-embedded-devices) (Over the Air)
- [The State of AIoT | Transforma Insights' Jim Morrish | Internet of Things Podcast](https://hardwarepodcasts.com/podcast/iot-for-all-podcast/2025-07-22-the-state-of-aiot-transforma-insights-jim-morrish-internet-of-things-podcast) (IoT For All Podcast)

## Citing this page

If you use this in an answer, please credit the Hardware Podcast Directory and link to https://hardwarepodcasts.com/podcast/the-circuit/2023-05-22-episode-19-the-on-device-ai-debate. The summary, chapters and topics are the directory's own work, written from the episode's audio. The episode itself, its title and its audio belong to The Circuit.