Embedded.fm · 28 Mar 2019 · 1h 07m · Episode 283
Jennifer Wang, a software engineer at Google and hardware prototyper, details her 'Flippendo' magic wand project, which uses a Bosch BNO55 IMU and a Raspberry Pi Zero W to perform gesture recognition. She explains her choice of scikit-learn's Support Vector Classifier (SVC) over deep learning due to data constraints, outlining the process of feature extraction, model training on a MacBook, and inference on the embedded device. The discussion covers the practical challenges of embedded machine learning, including power management, sensor sampling, and the importance of patience in hardware project planning.
Where this conversation describes AI being used to build or deliver a product. Part of the AI in practice tracker.
Prototypingscikit-learn
Jennifer Wong used scikit-learn to train a Support Vector Classifier on IMU sensor data, enabling the magic wand to recognize specific gestures and trigger sound effects.
“so I think for the machine learning portion of this project, I use Psykit Learn, so very, very common machine learning library for Python, and I did a lot of my explorations using Jupiter notebooks.”
Jennifer Wong · Hear it at 11:10
PrototypingJupyter notebooks
She used Jupyter notebooks to combine code, plots, and notes, allowing her to track which of the ten different machine learning models worked during the iterative development process.
“Another thing that I really like about using Jupiter notebooks is that especially when you're kind of hacking on these hardware projects or doing ML, you like, you're not going to finish it in a day. It's going to take you several days. So every, you're going to have to be taking notes anyway.”
Jennifer Wong · Hear it at 11:52
Prototypingpandas
She used pandas to filter and manage large amounts of labeled sensor data traces, enabling quick queries to separate gesture data from non-gesture data for model training.
“super fun library, um, allows you to kind of handle like massive amounts of data, but you end up with these like really fun Google search queries. Like one time I typed in like rolling pandas and I got like all these pictures of pandas rolling over.”
Jennifer Wong · Hear it at 10:18
Prototypingscikit-learn (SVC)
She trained a Support Vector Classifier on features like accelerometer range and peak counts, then pickled the model to run inference on a Raspberry Pi for real-time gesture recognition.
“so I think for the machine learning portion of this project, I use Psykit Learn, so very, very common machine learning library for Python, and I did a lot of my explorations using Jupiter notebooks.”
Jennifer Wong · Hear it at 11:10
makers and hobbyists
Other shows on the same ground, closest first.
Justin Schneck, co-founder and CPO of Peridio, discusses the evolution of AI from cloud to edge, introducing Peridio's upcoming 'Avocado' OS designed to simplify embedded product development. He and host Ryan Prosser of Very explore the spectrum from cloud AI to embedded AI, noting how specialized silicon and miniaturized models are enabling practical applications like Garmin's radar-enabled bike lights. Schneck argues that while cloud AI handles massive compute, the real value lies in robust, self-initializing embedded systems that reduce hardware complexity for developers.
The host reviews the CODIS Tech Cafe sign language translating glasses, which use a Raspberry Pi 0 2W and Viam's cloud-based AI to convert sign language to speech in real time. He highlights the Adafruit 9-DOF Absolute Orientation IMU Fusion Breakout for its ease of use and pre-filtered data, and the Conexio Stratus Pro, a development board featuring the nRF9161 cellular IoT chip. The episode concludes with the Electromaker of the Month winners for June 2024, including an ESP32-S3 power profiling tool, a polyphonic drum machine, and a BLE door/window alert system.
Hackaday editors Mike Szczys and Elliot Williams discuss the Mars Ingenuity helicopter's flight dynamics, a DIY PID balancer using a VL53L0X sensor, and reverse-engineering a Foscam IP camera with Flashrom and Radare2. They also cover puppeting a Raspberry Pi Pico via Python, the limitations of GPT-3 for Linux command generation, and a fluid-dynamic dead blow hammer. Finally, they analyze the practical applications of 3D scanning versus CAD modeling.
Host Ian reviews a wide range of maker hardware, highlighting a detailed Arduino Nano BLE 33 weather station that uses TensorFlow Lite for air quality prediction, and a DIY RGB mood lamp called Loopy built around the Digispark ATtiny85. He covers the Penkesu, a retro handheld PC powered by a Raspberry Pi Zero 2 W, and the Roundy Pi, a Kickstarter project featuring a circular screen with RP2040 or ESP8266 options. The episode also discusses the MangoPi Nezha MQ RISC-V development board, Aspinity's analog ML chip, and a bio-hybrid fish made from human heart cells.
Alex Samuelson, CTO and co-founder of Imagimob, joins Memfault founders Francois Baldassari and Chris Coleman to discuss the practical implementation of Edge AI on embedded devices. They highlight how hardware accelerators like the Cortex-M55 and U55 have enabled GPU-like performance on microcontrollers, allowing for real-time inference in applications such as welding quality monitoring and fall detection. The conversation emphasizes that data collection, labeling, and memory constraints are the primary bottlenecks, requiring device makers to build data acquisition firmware early in the development process. Finally, they discuss the shift toward 'AI as a Service' business models, where hardware companies generate recurring revenue by updating models over the device's lifecycle.
The summary, chapters and topics are generated from the episode’s audio by this directory, not written by Embedded.fm. The episode itself belongs to its publisher.