Why Testing Sucks

The Pragmatic Embedded Podcast · 11 May 2021 · 37m

What this episode covers

Jeff Gable and Luca discuss why testing feels difficult, arguing that the problem is often trying to test low-risk hardware features rather than high-risk software logic. They emphasize using risk matrices (probability vs. severity) to prioritize testing, advocating for hardware abstraction layers and re-architecting systems to avoid unpredictable interrupt interactions in safety-critical applications. The hosts also address how to approach testing legacy code, suggesting characterization tests and refactoring based on gut feeling about where bugs might lurk.

Mentioned

Companies
Agile Embedded Academy
Products
Embedded AI Podcast
Also
Working Effectively with Legacy Code, IEC 62304

Other shows on the same ground, closest first.

  • Embedded.fm · 56m

    30: Eventually Lightning Strikes

    James Grenning, author of 'Test Driven Development for Embedded C' and founder of Renaissance Software, explains how TDD reduces bugs and improves design in embedded systems. He argues that manual testing is unsustainable because the cost of retesting grows exponentially, leading to hidden defects that cause late-stage failures. Grenning advocates for getting code off the target hardware to run on PCs using mocks and fakes, which forces modularity and allows engineers to build confidence before hardware is available. He emphasizes that TDD is not just about testing but about driving SOLID design principles, such as single responsibility, by making complex code difficult to test.

  • Embedded.fm · 1h 13m

    290: Rule of Thumbs

    Phillip Johnston, founder of Embedded Artistry, discusses embedded software consulting, the importance of upfront design to avoid costly rewrites, and best practices for code reviews and testing. He analyzes the Boeing 737 MAX crashes as a systemic failure driven by business pressures and poor risk analysis, and explains the challenges of retargeting C++ standard libraries like libc++ to RTOS environments such as FreeRTOS and ThreadX.

  • Embedded.fm · 57m

    30: Eventually Lightning Strikes (Repeat)

    James Grenning, author of 'Test-Driven Development for Embedded C' and founder of Renaissance Software, explains how Test-Driven Development (TDD) improves embedded software engineering by enabling developers to write portable, modular code before hardware is available. He argues that TDD forces early design decisions, such as modularity and interface segregation, which reduces integration bugs and allows engineers to test logic on a PC rather than waiting for target hardware. Grenning also discusses his teaching methodology, emphasizing that engineers should experience TDD themselves to understand its value in reducing late-stage debugging and managing legacy code.

  • Embedded.fm · 1h 10m

    270: Broccoli Is Good Too

    James Grenning, author of 'Test Driven Development for Embedded C' and founder of Wingman Software, discusses the philosophy and practice of Test Driven Development (TDD) for embedded systems. He explains how TDD reduces debugging time by enforcing small, verifiable steps and using techniques like 'ZOMBIES' to defer complex decisions. Grenning also shares insights from a recent side project involving a Synapse Wireless mesh network for industrial water pressure monitoring, illustrating how to handle hardware uncertainty through experimentation and loose coupling.

  • Embedded.fm · 1h 03m

    53: Being a Grownup Engineer

    Jack Ganssle, author of 'The Art of Designing Embedded Systems' and long-time embedded industry expert, argues that firmware engineers must adopt disciplined engineering practices rather than relying on 'heroic' hacking. He discusses the necessity of requirements gathering, design reviews, code inspections, and rigorous testing to ensure safety and reliability, citing the Toyota unintended acceleration case as a failure of process. Ganssle advocates for measuring metrics like bug rates and lines of code to benchmark performance, and critiques the misuse of Linux in deeply embedded systems and the confusion between the maker movement and professional engineering.

The summary, chapters and topics are generated from the episode’s audio by this directory, not written by The Pragmatic Embedded Podcast. The episode itself belongs to its publisher.