Hardware Podcasts

141: Malevolent and Trying to Trick You

Embedded.fm · 2 Mar 2016 · 58m

What this episode covers

Julia Evans, a software engineer at Stripe, discusses using Linux system-level tools like `strace`, `perf`, and `top` to debug applications as black boxes. She explains how `strace` reveals system calls to understand program behavior without reading source code, and how `perf top` identifies CPU-intensive functions like garbage collection. Evans also shares her philosophy on debugging, emphasizing the importance of maintaining confidence and understanding that computers are logical machines, even when bugs seem impossible.

Most relevant to

early-stage companies

Topics

Mentioned

Products
Chrome
Tools
strace, perf, top, perf top, pkill, DTrace, Instruments
Also
Python, Java, C, Ruby, Go, NumPy, MAT Lab, LD_PRELOAD, PyCon, Bang Bang Con

Related episodes from other shows

Other shows on the same ground, closest first.

  • Oxide and Friends · 1h 06m

    What's a bug? What's a debugger?

    Bryan Cantrill and Adam Leventhal of Oxide Computer Company discuss the philosophy and mechanics of debugging, arguing that the term 'debugger' is regrettable and that tools should focus on observability and understanding system behavior rather than just fixing reproducible errors. They critique traditional in-situ breakpoint debugging for its fragility ('killing the patient') and advocate for post-mortem analysis, memory forensics, and tools like DTrace and Hubris. The conversation highlights how modern practices like CI/CD and languages like Rust have shifted the debugging landscape, making traditional debuggers less relevant for simple errors but more critical for complex, distributed system pathologies.

    Debugging & failure analysisEmbedded firmware
    10 chapters

  • The Pragmatic Embedded Podcast · 46m

    Linux Profiling with Mohammed Billoo

    Mohammed Billoo, founder of Mab Labs Embedded Solutions and author of the Embedded Linux Essentials Handbook, explains how to profile embedded Linux systems by distinguishing between VM size and VM RSS to identify memory leaks in scientific instruments. He details how to use tools like HTOP, perf, and LTTNG to prove that the CPU is rarely the bottleneck, often pointing to crypto engines, DMA, or file system I/O as the real constraints. Billoo advocates for baking kernel configurations like kallsyms and function tracers into builds from day one to enable rapid debugging without premature optimization. The conversation also covers using LLMs to interpret complex tool outputs like Valgrind and perf, while emphasizing that physical measurements remain the ultimate arbiter of system performance.

    Embedded firmwareDebugging & failure analysis
    11 chapters

  • Oxide and Friends · 1h 05m

    from /proc to proc_macro

    Bryan Cantrill, Adam Leventhal, and other former Sun Microsystems engineers discuss the history and technical challenges of debugging infrastructure, focusing on Roger Faulkner's invention of the /proc filesystem and the subsequent development of ptrace and DTrace. They detail the complex emulation of Linux branded zones (LX) in Solaris, explaining how the need to emulate ptrace and handle stack switching for Go led to significant debugging pain points and 'magic' that did not layer well. The conversation also covers the design philosophy behind DTrace, its inspiration from the OCaml language, and the modern application of these concepts in Rust's proc macros for creating domain-specific languages with robust error handling.

    Debugging & failure analysisEmbedded firmware
    8 chapters

  • Oxide and Friends · 1h 35m

    A Debugging Odyssey

    Dave Pacheco of Oxide Computer Company details his months-long debugging odyssey to resolve a critical memory corruption bug in the Go runtime on Illumos. He explains how he used DTrace, MDB, and binary patching to identify that the OS failed to preserve YMM registers during signal handling, a flaw that could have caused catastrophic data corruption in their CockroachDB control plane. The episode highlights the importance of rigorous documentation, deep tooling mastery, and the cultural value of tackling seemingly unreproducible low-level failures.

    Debugging & failure analysisEmbedded firmware
    9 chapters

  • Oxide and Friends · 1h 47m

    Paths into Systems Programming

    Bryan Cantrill, Adam Leventhal, and other engineers from Oxide Computer Company discuss the nature of systems programming, defining it as software that runs, creates, or debugs other software. The group shares personal anecdotes about entering the field, highlighting the importance of curiosity, rigor, and contributing to open source communities like Joyent and KVM. They emphasize that breaking into systems programming often requires demonstrating deep technical interest through tangible artifacts, such as code contributions or reverse engineering projects, rather than relying solely on traditional credentials.

    Careers & educationOpen-source hardware
    6 chapters

The summary, chapters and topics are generated from the episode’s audio by this directory, not written by Embedded.fm. The episode itself belongs to its publisher.