Hardware Podcasts

38: Blame the Monkey

Embedded.fm · 12 Feb 2014 · 1h 01m

What this episode covers

Elecia White and producer Chris White discuss strategies for debugging 'insurmountable' problems in embedded systems, ranging from race conditions and stack overflows to non-reproducible hardware issues. They advocate for methodical approaches such as drawing system diagrams, writing detailed emails to clarify thoughts, and using the scientific method to isolate variables. The conversation also covers the importance of logging, reverting to known good code, and knowing when to walk away or accept a bug as a feature.

Most relevant to

established companies

Topics

Mentioned

Also
Stack Exchange, DMA, FFT, Netflix Chaos Monkey

Related episodes from other shows

Other shows on the same ground, closest first.

  • Oxide and Friends · 1h 30m

    Debugging Methodologies

    Jordan Hendricks and Luqman Aden of Oxide Computer Company detail their methodologies for debugging complex hardware and compiler issues. Hendricks explains how he used the KMDB in-situ debugger to diagnose an NVMe reset hang, discovering that switching from abrupt to normal shutdown notifications resolved the issue. Aden describes his process for debugging a rustc segfault involving inline assembly in async blocks, using LLVM IR analysis and Rust Analyzer to isolate a bug in the compiler's code generation. The episode also covers a third case involving a triple fault caused by undefined behavior in a bare-metal bootloader, highlighting the importance of tooling and collaborative debugging culture at Oxide.

    Debugging & failure analysisEmbedded firmware
    5 chapters

  • Oxide and Friends · 1h 35m

    A Debugging Odyssey

    Dave Pacheco of Oxide Computer Company details his months-long debugging odyssey to resolve a critical memory corruption bug in the Go runtime on Illumos. He explains how he used DTrace, MDB, and binary patching to identify that the OS failed to preserve YMM registers during signal handling, a flaw that could have caused catastrophic data corruption in their CockroachDB control plane. The episode highlights the importance of rigorous documentation, deep tooling mastery, and the cultural value of tackling seemingly unreproducible low-level failures.

    Debugging & failure analysisEmbedded firmware
    9 chapters

  • The Amp Hour · 1h 01m

    Rubber Ducking

    Dave Jones and Chris Gammell discuss engineering troubleshooting methodologies, including 'rubber ducking' and the importance of engineering notebooks, illustrated by Chris's debugging of an STM32 timer using GDB. They examine the market for low-cost electronics from China, with Dave Jones analyzing an $8.70 precision instrument found on AliExpress, and discuss the history and current state of companies like Motorola and Atari. The hosts also cover Jerry Ellsworth's new Kickstarter for the Cast AR augmented reality headset and the commercial availability of Boston Dynamics' Spot robot.

    Debugging & failure analysisSourcing & supply chain
    10 chapters

  • The Pragmatic Embedded Podcast · 56m

    Modern Embedded Systems Programming with Miro Samek

    Miro Samek, founder and CEO of Quantum Leaps, explains why traditional embedded architectures like superloops and RTOS lead to 'spaghetti code' and concurrency bugs, advocating instead for event-driven active objects and hierarchical state machines. He details how this approach eliminates race conditions and mutexes by enforcing a 'no sharing' principle, thereby improving testability and development velocity. Samek also discusses the importance of using lightweight modeling tools to generate code from state diagrams, ensuring the model remains the source of truth in an agile workflow.

    Embedded firmwareDebugging & failure analysis
    8 chapters

  • The Amp Hour · 1h 44m

    Pedantic or Andrantic

    Chris Gammell, a developer advocate at Hologram and founder of Contextual Electronics, joins Elecia and Chris White of Embedded.fm to discuss hardware prototyping, learning strategies, and industry tools. The conversation covers the challenges of cellular IoT connectivity, the utility of the HackRF software-defined radio, and debugging techniques using the J-Trace Pro. They also explore pedagogical concepts like the Dreyfus model for learning programming and hardware, emphasizing the importance of iteration and mental health in engineering careers.

    Embedded firmwareDebugging & failure analysis
    7 chapters

The summary, chapters and topics are generated from the episode’s audio by this directory, not written by Embedded.fm. The episode itself belongs to its publisher.