EE Times Current · 1 Oct 2021 · 40m
Steven Woo, a Rambus fellow and distinguished inventor, explains how AI's exponential growth in model size and training data is creating unprecedented demands on memory capacity, bandwidth, and power efficiency. He details the industry's shift toward High Bandwidth Memory (HBM) and GDDR to handle these loads, while noting that emerging memories like ReRAM and MRAM face reliability hurdles. Woo also discusses how architectural innovations like Compute Express Link (CXL) and chip stacking are critical for reducing the power and latency associated with moving data between processors and memory.
Where this conversation describes AI being used to build or deliver a product. Part of the AI in practice tracker.
Design
AI is being used to design self-synthesizing networks that design their own components and evolve automatically, rather than humans designing networks that machines merely configure.
“what about replacing the design of the network? But with AI. So not humans doing this, but actually artificial intelligence starting to design its own network components.”
Speaker 3 · Hear it at 5:57
large enterprises
Other shows on the same ground, closest first.
Jeremy Werner, SVP and GM of Micron’s Core Data Center Business Unit, explains how AI is shifting the memory industry from a cyclical model to sustainable growth driven by the 'memory wall' in inference. He details Micron's strategy to break this bottleneck through innovations like HBM4 and ultra-high-capacity SSDs, which address critical data center constraints of power and physical space. The conversation covers the technical hierarchy of memory, from HBM to context storage, and Micron's aggressive capacity expansion plans to meet surging demand for AI training and inference workloads.
Steven Woo, a Fellow at Rambus, explains the fundamental differences between memory and logic, detailing how DRAM innovation is measured by capacity and bandwidth rather than process nodes. He discusses the specific challenges of High Bandwidth Memory (HBM) for AI training, including stacking and power efficiency, and contrasts this with the cost constraints of standard DDR. The conversation covers the massive capital expenditures required for memory fabs, the role of chiplets in increasing package complexity, and the emerging competitive landscape involving Chinese manufacturer CXMT.
Ben Bajarin and Jay Goldberg analyze AMD's recent earnings, noting that while the execution was solid and data center CPU demand doubled year-over-year, the market reaction was muted due to conservative guidance and high capital expenditure expectations. Ben reports from the Future of Memory Summit, highlighting the maturation of the CXL (Compute Express Link) ecosystem, driven by hyperscalers' need to reuse legacy DDR4 memory for inference workloads. The hosts also discuss the durability of long-term agreements (LTAs) in the memory supercycle, citing Sandisk's strong margins, and identify SiTime as a key beneficiary of the rack-scale AI infrastructure boom.
Ben Bajarin and Jay Goldberg analyze the 'boom-bust-build-out' cycle in the current AI hardware market, arguing that while speculative excesses will correct, they leave behind productive infrastructure like power grids and cheap compute. They examine ARM's strategic shift toward first-party chip design and its acquisition of Dream Big Semiconductors to enhance CPU capabilities, alongside Qualcomm's growth driven by premium Android ASPs and its entry into data center inference. The hosts also discuss supply chain dynamics, noting how Tesla and OpenAI are securing foundry capacity through aggressive negotiations, while memory and storage companies like SanDisk manage tight supply through measured capacity expansion.
Ben Bajarin and Jay Goldberg discuss the semiconductor memory market, noting that Chinese providers YMTC and CXMT are ramping capacity and improving yields, which may offer pricing relief to OEMs squeezed by data center demand. They analyze Meta's new 'Muse' AI assistant as a potential catalyst for mainstream adoption by bridging the user skill gap through proactive orchestration, while also highlighting the massive, unforeseen surge in CPU demand required to power these continuous 'agentic' workloads. Finally, they unpack Anthropic's $11 billion equity-exchange partnership with Akamai, highlighting its implications for distributed edge inference and the critical need for CPU infrastructure.
The summary, chapters and topics are generated from the episode’s audio by this directory, not written by EE Times Current. The episode itself belongs to its publisher.