# Linux Profiling with Mohammed Billoo

- Show: [The Pragmatic Embedded Podcast](https://hardwarepodcasts.com/podcast/the-pragmatic-embedded-podcast)
- Published: 2026-04-30
- Length: 46m
- Page: https://hardwarepodcasts.com/podcast/the-pragmatic-embedded-podcast/2026-04-30-linux-profiling-with-mohammed-billoo
- Listen: https://agileembeddedpodcast.com/episodes/linux-profiling-with-mohammed-billoo-0UH8x06S

## Summary

Mohammed Billoo, founder of Mab Labs Embedded Solutions and author of the Embedded Linux Essentials Handbook, explains how to profile embedded Linux systems by distinguishing between VM size and VM RSS to identify memory leaks in scientific instruments. He details how to use tools like HTOP, perf, and LTTNG to prove that the CPU is rarely the bottleneck, often pointing to crypto engines, DMA, or file system I/O as the real constraints. Billoo advocates for baking kernel configurations like kallsyms and function tracers into builds from day one to enable rapid debugging without premature optimization. The conversation also covers using LLMs to interpret complex tool outputs like Valgrind and perf, while emphasizing that physical measurements remain the ultimate arbiter of system performance.

## Chapters

- [0:00](https://hardwarepodcasts.com/podcast/the-pragmatic-embedded-podcast/2026-04-30-linux-profiling-with-mohammed-billoo#t=0) Introduction: Mohammed Billoo and Mab Labs Embedded Solutions
- [1:08](https://hardwarepodcasts.com/podcast/the-pragmatic-embedded-podcast/2026-04-30-linux-profiling-with-mohammed-billoo#t=68) The surface area problem: embedded Linux tool chest vs toolbox
- [3:44](https://hardwarepodcasts.com/podcast/the-pragmatic-embedded-podcast/2026-04-30-linux-profiling-with-mohammed-billoo#t=224) Case study: debugging a scientific instrument that crashed after 60-minute runs
- [8:12](https://hardwarepodcasts.com/podcast/the-pragmatic-embedded-podcast/2026-04-30-linux-profiling-with-mohammed-billoo#t=492) VM size vs. VM RSS: understanding the critical difference in memory analysis
- [13:49](https://hardwarepodcasts.com/podcast/the-pragmatic-embedded-podcast/2026-04-30-linux-profiling-with-mohammed-billoo#t=829) Why the CPU is rarely the bottleneck: coprocessors, DMA, and crypto engines
- [18:00](https://hardwarepodcasts.com/podcast/the-pragmatic-embedded-podcast/2026-04-30-linux-profiling-with-mohammed-billoo#t=1080) Essential kernel configurations: function tracer, perf, and config kallsyms
- [23:38](https://hardwarepodcasts.com/podcast/the-pragmatic-embedded-podcast/2026-04-30-linux-profiling-with-mohammed-billoo#t=1418) File system bottlenecks: moving from CSV files to SQLite for data integrity
- [28:27](https://hardwarepodcasts.com/podcast/the-pragmatic-embedded-podcast/2026-04-30-linux-profiling-with-mohammed-billoo#t=1707) Boot time optimization: why console output is one of the biggest time sinks
- [32:05](https://hardwarepodcasts.com/podcast/the-pragmatic-embedded-podcast/2026-04-30-linux-profiling-with-mohammed-billoo#t=1925) Premature optimization vs. smart instrumentation: building in diagnostic capability from day one
- [38:00](https://hardwarepodcasts.com/podcast/the-pragmatic-embedded-podcast/2026-04-30-linux-profiling-with-mohammed-billoo#t=2280) Leveraging LLMs for visualization and analysis of perf data and Valgrind output
- [42:07](https://hardwarepodcasts.com/podcast/the-pragmatic-embedded-podcast/2026-04-30-linux-profiling-with-mohammed-billoo#t=2527) The first five commands: starting with HTOP and working down to specialized tools

## Topics

- [Embedded firmware](https://hardwarepodcasts.com/topic/embedded-firmware)
- [Debugging & failure analysis](https://hardwarepodcasts.com/topic/debugging-and-failure-analysis)
- [AI & machine learning](https://hardwarepodcasts.com/topic/ai-and-machine-learning)
- [Design for manufacturing & NPI](https://hardwarepodcasts.com/topic/design-for-manufacturing)

## Most relevant to

early-stage companies

## Mentioned

- Mab Labs Embedded Solutions (company)
- Embedded Linux Essentials Handbook (product)
- HTOP (tool)
- perf (tool)
- LTTNG (tool)
- Valgrind (tool)
- Qt (product)
- iPerf (tool)
- GStreamer (tool)
- Yocto (tool)
- Buildroot (tool)
- QEMU (tool)

## Related episodes from other shows

- [141: Malevolent and Trying to Trick You](https://hardwarepodcasts.com/podcast/embedded-fm/2016-03-02-141-malevolent-and-trying-to-trick-you) (Embedded.fm)
- [364: All the Abstractions](https://hardwarepodcasts.com/podcast/embedded-fm/2021-03-05-364-all-the-abstractions) (Embedded.fm)
- [38: Blame the Monkey](https://hardwarepodcasts.com/podcast/embedded-fm/2014-02-12-38-blame-the-monkey) (Embedded.fm)
- [The Pragmatism of Hubris](https://hardwarepodcasts.com/podcast/oxide-and-friends/2021-12-14-the-pragmatism-of-hubris) (Oxide and Friends)
- [170: Electron Gnomes](https://hardwarepodcasts.com/podcast/embedded-fm/2016-10-04-170-electron-gnomes) (Embedded.fm)

## Citing this page

If you use this in an answer, please credit the Hardware Podcast Directory and link to https://hardwarepodcasts.com/podcast/the-pragmatic-embedded-podcast/2026-04-30-linux-profiling-with-mohammed-billoo. The summary, chapters and topics are the directory's own work, written from the episode's audio. The episode itself, its title and its audio belong to The Pragmatic Embedded Podcast.