Hardware Podcasts

Adversarial Machine Learning

Oxide and Friends · 27 Mar 2024 · 1h 23m · Episode 8

What this episode covers

Nicholas Carlini, a security researcher specializing in adversarial machine learning, joins Bryan Cantrill and Adam Leventhal to discuss how sequences of seemingly random tokens can cause Large Language Models (LLMs) to ignore safety restrictions. Carlini explains that these attacks, which include 'poisoning' training data by exploiting expired domain names and using adversarial suffixes to bypass jailbreaks, demonstrate that LLMs are vulnerable to inputs that look like noise but trigger specific internal features. The conversation highlights the transferability of these attacks across different models, the importance of open-source models for security research, and the practical risks of deploying LLMs without understanding their failure modes.

Most relevant to

early-stage companies

Topics

Mentioned

Companies
Oxide Computer Company, OpenAI
Products
GPT-4, LLaMA, Stable Diffusion
Also
Nicholas Carlini, Bryan Cantrill, Adam Leventhal, ImageNet, CIFAR-10, Spectre, Meltdown, Return Oriented Programming, printf, GANs, Ian Goodfellow, Zico Kolter, Matt Fredrikson, Andy Ziegle, Alexander Madry, Nicola Papernot, Dave Wagner, Simon Wilson, Philip Ball

Related episodes from other shows

Other shows on the same ground, closest first.

  • Embedded.fm · 1h 08m

    502: Chat, J'ai Peté!

    Elecia White and Christopher White discuss the TV adaptation of Martha Wells' 'Murderbot Diaries' and the deceptive nature of Large Language Models (LLMs) like ChatGPT, Claude, and Gemini. They highlight specific failures of AI, including lawyers citing non-existent legal precedents and GitHub Copilot agents generating endless, looping pull requests. The conversation also covers control systems, contrasting theoretical 'fancy math' like Kalman filters with empirical tuning, and mentions the Segway patent US7370713B1.

    AI & machine learningEmbedded firmware
    6 chapters

  • IoT For All Podcast · 20m

    Large Language Models and How To Use Them | Cyrano.ai's Scott Sandland | Internet of Things Podcast

    Scott Sandland, Founder and CEO of Cyrano.ai, explains that Large Language Models (LLMs) are essentially advanced autocomplete systems that predict the next word based on massive datasets, making natural language the new interface for computing. He argues that while LLMs accelerate execution, the 'move fast and break things' philosophy is dangerous for AI due to unpredictable ethical outcomes and biases, necessitating a parallel 'ethics roadmap' alongside technical development. Sandland highlights Cyrano.ai's proprietary API, which uses linguistic analysis to assess empathy and relationship value in contexts like sales and mental health, rather than relying on generic sentiment analysis.

    AI & machine learningSecurity & privacy
    6 chapters

  • Global Medical Device Podcast · 53m

    #464: Why LLM Unpredictability is a Liability in MedTech

    Tyler Harmon, CEO of Iaso Automated Medical Systems, explains why the inherent randomness in Large Language Models (LLMs) makes them unsuitable for safety-critical medical devices compared to deterministic algorithms like Convolutional Neural Networks (CNNs). He details how Iaso uses non-LLM machine learning to detect Acute Respiratory Distress Syndrome (ARDS) in ICU patients and advocates for treating third-party AI stacks as Software of Unknown Provenance (SOUP). Harmon also outlines the need for a dedicated Machine Learning Management System (AIMS) within Quality Management Systems to govern AI governance, hallucination risks, and data integrity.

    AI & machine learningQuality & yield
    7 chapters

  • The Pragmatic Embedded Podcast · 55m

    Crossover with Embedded AI Podcast

    Jeff Gable, Luca, and Ryan Torvik discuss integrating Large Language Models (LLMs) into embedded systems engineering workflows, emphasizing that AI must augment rather than replace rigorous engineering discipline. They identify 'vibe coding' as a critical anti-pattern where developers lose control of architectural decisions by letting AI generate too much code at once, leading to unmanageable outputs and hallucinations. The hosts advocate for strict context management, test-driven development (TDD), and small, incremental commits to maintain deterministic quality, particularly in safety-critical domains like medical devices and automotive systems. They conclude that while LLMs accelerate prototyping and boilerplate generation, human code review and ownership remain non-negotiable for reliable production software.

    Embedded firmwareDebugging & failure analysis
    8 chapters

  • The Next Byte Podcast · 34m

    88. Dall-E, But Better

    Hosts Daniel and Farbode discuss OpenAI's DALL-E 2, explaining how it uses CLIP embeddings to generate photorealistic images from text prompts. They analyze DALL-E's limitations with spatial relationships and text rendering, and explore the ethical concerns regarding training data bias and artist copyright. The episode highlights MIT research into 'composable diffusion,' a method that combines multiple AI models to improve accuracy and creativity in generated imagery.

    AI & machine learningSecurity & privacy
    6 chapters

The summary, chapters and topics are generated from the episode’s audio by this directory, not written by Oxide and Friends. The episode itself belongs to its publisher.