Hardware Podcasts

192. Karsten Roscher & The Need For Safe AI (Interview)

The Next Byte Podcast · 17 Sept 2024 · 1h 02m

What this episode covers

Karsten Roscher, Head of the Certifiable AI department at the Fraunhofer Institute for Cognitive Systems (IKS), discusses the critical need for safety in AI development, particularly in autonomous driving. He explains the challenges of out-of-distribution detection, where models fail on unseen data like e-scooters or horse carriages, and introduces Robuscope, a free tool for evaluating model robustness without sharing proprietary data. Roscher also critiques the industry's over-reliance on benchmarks and highlights a new SAE International course designed to bridge the gap between machine learning engineers and safety experts.

Most relevant to

established companies

Topics

Mentioned

Companies
Fraunhofer Institute for Cognitive Systems IKS, OpenAI, Tesla
Tools
Robuscope

Related episodes from other shows

Other shows on the same ground, closest first.

  • EE Times Current · 39m

    Arm’s bold new path | Teenage autonomous drivers | Dealing with the data deluge

    Kevin Krewell of TIRIAS Research reports on Arm Techcon, highlighting Arm's decision to open its instruction set architecture (ISA) to customer-defined custom instructions, a strategic shift driven by CEO Sophie Richardson and SoftBank's influence to compete with RISC-V. Phil Koopman of Carnegie Mellon University discusses the challenges of certifying autonomous vehicles, arguing that current road tests are insufficient for AI systems lacking common sense, and explains how the UL 4600 standard requires manufacturers to prove they have mitigated risks through rigorous engineering evidence rather than just passing a driving test. John Petty of John Petty Research joins to discuss the 'data deluge,' explaining how data visualization and AI tools are essential for making sense of massive datasets from IoT, simulations, and financial transactions.

    Chips, FPGAs & semiconductorsSecurity & privacy
    6 chapters

  • The Robot Report Podcast · 1h 04m

    APEX.AI discusses autonomous vehicle OS's; Nearly $1B in funding announcements

    Jan Becker, founder and CEO of APEX.AI, and Joe Speed, CTO of APEX.AI, discuss the evolution of real-time operating systems for autonomous vehicles, specifically how APEX.AI built a safety-certified, real-time version of ROS 2. They detail the company's involvement in the Indy Autonomous Challenge, explaining how their software stack enables high-speed sensor fusion and processing for race cars reaching speeds over 175 mph. The episode also covers APEX.AI's recent Series B funding and its expansion beyond automotive into agriculture and aerial robotics.

    Embedded firmwareMicrocontrollers & dev platforms
    7 chapters

  • Embedded.fm · 1h 16m

    421: Paint the Iceberg Yellow

    Chris Hobbs, who works in kernel development at BlackBerry QNX and authored 'Embedded Software Development for Safety-Critical Systems,' explains the distinction between safety-critical and standard embedded systems, emphasizing safety culture and the concept of SOTIF (Safety of the Intended Functionality). He details how modern systems, such as autonomous vehicles and medical devices, face risks from 'accidental systems' where interactions between components create unforeseen hazards, citing examples like a radar system dependent on jammed GPS. Hobbs advocates for goal-based standards like UL 4600 and the use of Safety Cases with Goal Structuring Notation to demonstrate safety rather than relying on prescriptive checklists. He also discusses the challenges of verifying machine learning algorithms, the limitations of languages like C and Rust in safety contexts, and the importance of structured risk analysis using tools like Bayesian fault trees and LaTeX for traceability.

    Embedded firmwareSecurity & privacy
    12 chapters

  • Robot Talk · 34m

    Episode Eleven – Self-Driving Cars: behind the wheel

    Dr Richard Hawkins of the University of York and Dr Frederic Labrosse of Aberystwyth University discuss the distinct challenges of autonomous vehicle safety, regulation, and perception. Hawkins explains the difficulty of defining safety cases for AI-driven cars in complex urban environments and the UK's new regulations for automated lane keeping systems. Labrosse contrasts on-road challenges, like plastic bag detection, with off-road issues such as terrain variability and LIDAR sensitivity to grass, highlighting applications in farming, archaeology, and surveying.

    Robotics & autonomySensors & measurement in products
    9 chapters

  • EE Times Current · 30m

    A Disaster Waiting to Happen | Get Up, Stand Up

    Phil Copelman, co-founder of Edge Case Research, argues that SAE Level 3 autonomy is a safety disaster because the standard's complex handoff requirements are unintelligible to drivers, creating a dangerous gap between engineering definitions and human understanding. The episode also examines Amazon's labor practices through the lens of Robin Gaster, author of 'Behemoth, Amazon Rising,' discussing the unionization vote at the Bessemer, Alabama facility and the company's culture of worker disposability. Additionally, the hosts cover Arm's announcement of its V9 architecture and Boston Dynamics' new 'Stretch' warehouse robot.

    Debugging & failure analysisHiring & workforce
    5 chapters

The summary, chapters and topics are generated from the episode’s audio by this directory, not written by The Next Byte Podcast. The episode itself belongs to its publisher.