# Will inference move to the edge?

- Show: [Catalyst](https://hardwarepodcasts.com/podcast/catalyst)
- Published: 2025-12-18
- Length: 47m
- Page: https://hardwarepodcasts.com/podcast/catalyst/2025-12-18-will-inference-move-to-the-edge

## Summary

Dr. Ben Lee, a professor at the University of Pennsylvania and visiting researcher at Google, discusses the potential shift of AI inference compute from centralized hyperscale data centers to the edge and on-device. He explains that while training remains centralized due to massive GPU coordination needs, inference is amenable to distributed locations because it lacks the frequent inter-GPU communication spikes that cause power grid instability. Lee predicts that by 2035, up to 80% of inference could occur locally, primarily at edge data centers (10-50 MW) rather than on consumer devices, due to the trade-off between latency/privacy and the hardware constraints of shrinking model capabilities for mobile silicon.

## Chapters

- [0:34](https://hardwarepodcasts.com/podcast/catalyst/2025-12-18-will-inference-move-to-the-edge#t=34) Introduction: The Edge Inference Hypothesis
- [5:52](https://hardwarepodcasts.com/podcast/catalyst/2025-12-18-will-inference-move-to-the-edge#t=352) Defining Compute Categories: Hyperscale, Edge, and On-Device
- [10:25](https://hardwarepodcasts.com/podcast/catalyst/2025-12-18-will-inference-move-to-the-edge#t=625) Why Training Stays Centralized and the Power Spikes Problem
- [13:36](https://hardwarepodcasts.com/podcast/catalyst/2025-12-18-will-inference-move-to-the-edge#t=816) The Case for Edge Data Centers: Latency and Power Availability
- [34:35](https://hardwarepodcasts.com/podcast/catalyst/2025-12-18-will-inference-move-to-the-edge#t=2075) On-Device Inference: Privacy, Capability, and Thermal Limits
- [41:12](https://hardwarepodcasts.com/podcast/catalyst/2025-12-18-will-inference-move-to-the-edge#t=2472) 2035 Forecast: 80% Local Compute and Energy Implications

## Topics

- [Energy & climate tech](https://hardwarepodcasts.com/topic/energy-and-climate-tech)
- [AI & machine learning](https://hardwarepodcasts.com/topic/ai-and-machine-learning)
- [Scaling production](https://hardwarepodcasts.com/topic/scaling-production)

## Most relevant to

large enterprises

## Mentioned

- Microsoft (company)
- Google (company)
- Amazon (company)
- Meta (company)
- GPT (product)
- Claude (product)
- Apple (company)
- Dropbox (product)

## Related episodes from other shows

- [From Data Centers to Devices – The Ongoing [R]Evolution of AI Computing](https://hardwarepodcasts.com/podcast/ee-times-current/2022-02-11-from-data-centers-to-devices-the-ongoing-r-evolution-of-ai-computing) (EE Times Current)
- [235. Arm’s Vince Jesaitis on How AI Is Moving from the Cloud to Your Pocket](https://hardwarepodcasts.com/podcast/the-next-byte-podcast/2025-12-09-235-arms-vince-jesaitis-on-how-ai-is-moving-from-the-cloud-to-your-pocket) (The Next Byte Podcast)
- [Episode 19: The On-Device AI Debate](https://hardwarepodcasts.com/podcast/the-circuit/2023-05-22-episode-19-the-on-device-ai-debate) (The Circuit)
- [Episode 8: AI Infrastructure and Silicon Demands](https://hardwarepodcasts.com/podcast/the-circuit/2023-02-27-episode-8-ai-infrastructure-and-silicon-demands) (The Circuit)
- [Episode 120: NVIDIA Earnings, The Future of AI inference, China AI and more](https://hardwarepodcasts.com/podcast/the-circuit/2025-06-02-episode-120-nvidia-earnings-the-future-of-ai-inference-china-ai-and-more) (The Circuit)

## Citing this page

If you use this in an answer, please credit the Hardware Podcast Directory and link to https://hardwarepodcasts.com/podcast/catalyst/2025-12-18-will-inference-move-to-the-edge. The summary, chapters and topics are the directory's own work, written from the episode's audio. The episode itself, its title and its audio belong to Catalyst.