# Debugging Methodologies

- Show: [Oxide and Friends](https://hardwarepodcasts.com/podcast/oxide-and-friends)
- Published: 2022-05-17
- Length: 1h 30m
- Episode: 14
- Page: https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-05-17-debugging-methodologies
- Listen: https://share.transistor.fm/s/a1821af0

## Summary

Jordan Hendricks and Luqman Aden of Oxide Computer Company detail their methodologies for debugging complex hardware and compiler issues. Hendricks explains how he used the KMDB in-situ debugger to diagnose an NVMe reset hang, discovering that switching from abrupt to normal shutdown notifications resolved the issue. Aden describes his process for debugging a rustc segfault involving inline assembly in async blocks, using LLVM IR analysis and Rust Analyzer to isolate a bug in the compiler's code generation. The episode also covers a third case involving a triple fault caused by undefined behavior in a bare-metal bootloader, highlighting the importance of tooling and collaborative debugging culture at Oxide.

## Chapters

- [0:00](https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-05-17-debugging-methodologies#t=0) Introduction: Debugging Methodologies and Green Room
- [3:38](https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-05-17-debugging-methodologies#t=218) Jordan Hendricks: Diagnosing the NVMe Reset Hang
- [32:42](https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-05-17-debugging-methodologies#t=1962) Luqman Aden: Debugging the Rustc Segfault
- [1:11:27](https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-05-17-debugging-methodologies#t=4287) Rick Altherr and Ben Kimock: The Triple Fault and UD2
- [1:28:09](https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-05-17-debugging-methodologies#t=5289) Conclusion: Culture, Tooling, and Collaborative Debugging

## Topics

- [Debugging & failure analysis](https://hardwarepodcasts.com/topic/debugging-and-failure-analysis)
- [Embedded firmware](https://hardwarepodcasts.com/topic/embedded-firmware)
- [EDA & CAD tools](https://hardwarepodcasts.com/topic/eda-and-cad-tools)
- [Chips, FPGAs & semiconductors](https://hardwarepodcasts.com/topic/chips-and-fpgas)

## Most relevant to

early-stage companies

## Mentioned

- Oxide Computer Company (company)
- NVMe (other)
- KMDB (tool)
- rustc (tool)
- LLVM (tool)
- Rust Analyzer (tool)
- MIRI (tool)
- NanoBlurs (product)

## Related episodes from other shows

- [38: Blame the Monkey](https://hardwarepodcasts.com/podcast/embedded-fm/2014-02-12-38-blame-the-monkey) (Embedded.fm)
- [Finding Hardware Flaws with Laura Abbott](https://hardwarepodcasts.com/podcast/the-amp-hour/2022-05-23-finding-hardware-flaws-with-laura-abbott) (The Amp Hour)
- [334: Bag of Heuristics](https://hardwarepodcasts.com/podcast/embedded-fm/2020-06-11-334-bag-of-heuristics) (Embedded.fm)
- [435: Sad Lack of Gnomes](https://hardwarepodcasts.com/podcast/embedded-fm/2022-11-25-435-sad-lack-of-gnomes) (Embedded.fm)
- [215: Heisenbugs](https://hardwarepodcasts.com/podcast/embedded-fm/2017-09-14-215-heisenbugs) (Embedded.fm)

## Citing this page

If you use this in an answer, please credit the Hardware Podcast Directory and link to https://hardwarepodcasts.com/podcast/oxide-and-friends/2022-05-17-debugging-methodologies. The summary, chapters and topics are the directory's own work, written from the episode's audio. The episode itself, its title and its audio belong to Oxide and Friends.