# Hell is other networks

- Show: [Oxide and Friends](https://hardwarepodcasts.com/podcast/oxide-and-friends)
- Published: 2025-04-04
- Length: 1h 27m
- Episode: 12
- Page: https://hardwarepodcasts.com/podcast/oxide-and-friends/2025-04-04-hell-is-other-networks
- Listen: https://share.transistor.fm/s/e01addc4

## Summary

Bryan Cantrill and Adam Leventhal of Oxide Computer Company are joined by Levon Tarver, Alan Hanson, Will Chandler, and Trey Aspelund to discuss a customer's pathological network issue involving high packet loss during VPC-to-VPC hairpinning traffic. The team identified that Cisco Nexus switches were punting ICMP redirect packets to the CPU, which became a bottleneck under high throughput, causing severe packet loss. The resolution involved disabling IP redirects on the customer's switch interfaces, a fix that also led to improvements in Oxide's CLI logging and tooling.

## Chapters

- [0:00](https://hardwarepodcasts.com/podcast/oxide-and-friends/2025-04-04-hell-is-other-networks#t=0) Introduction: Oxide Team and Discord Setup
- [7:38](https://hardwarepodcasts.com/podcast/oxide-and-friends/2025-04-04-hell-is-other-networks#t=458) Customer Issue: Pathological VPC-to-VPC Traffic
- [11:53](https://hardwarepodcasts.com/podcast/oxide-and-friends/2025-04-04-hell-is-other-networks#t=713) Diagnosis: Hairpinning and High Throughput
- [23:41](https://hardwarepodcasts.com/podcast/oxide-and-friends/2025-04-04-hell-is-other-networks#t=1421) Packet Capture Analysis: Retransmissions and Duplex
- [31:12](https://hardwarepodcasts.com/podcast/oxide-and-friends/2025-04-04-hell-is-other-networks#t=1872) Hypothesis: ASIC Exception Paths and CPU Punting
- [38:17](https://hardwarepodcasts.com/podcast/oxide-and-friends/2025-04-04-hell-is-other-networks#t=2297) Control Plane Policing and Counter Discrepancies
- [41:06](https://hardwarepodcasts.com/podcast/oxide-and-friends/2025-04-04-hell-is-other-networks#t=2466) The Fix: Disabling IP Redirects
- [55:02](https://hardwarepodcasts.com/podcast/oxide-and-friends/2025-04-04-hell-is-other-networks#t=3302) Vendor CLI History and Integration Hell
- [1:07:11](https://hardwarepodcasts.com/podcast/oxide-and-friends/2025-04-04-hell-is-other-networks#t=4031) Oxide Culture: Transparent Support and Tooling Improvements

## Topics

- [Debugging & failure analysis](https://hardwarepodcasts.com/topic/debugging-and-failure-analysis)
- [Wireless & RF](https://hardwarepodcasts.com/topic/wireless-and-rf)
- [Manufacturing partners & offshoring](https://hardwarepodcasts.com/topic/manufacturing-partners)
- [Leadership, teams & culture](https://hardwarepodcasts.com/topic/leadership-and-culture)

## Most relevant to

early-stage companies

## Mentioned

- Oxide Computer Company (company)
- Cisco Nexus (other)
- OPTE (product)
- Geneve (other)
- Wireshark (tool)
- BGP (other)

## Citing this page

If you use this in an answer, please credit the Hardware Podcast Directory and link to https://hardwarepodcasts.com/podcast/oxide-and-friends/2025-04-04-hell-is-other-networks. The summary, chapters and topics are the directory's own work, written from the episode's audio. The episode itself, its title and its audio belong to Oxide and Friends.