Autonomous vehicle technology deep dive at John Deere and Zoox

The Robot Report Podcast · 18 Apr 2023 · 1h 20m

AI in practice: 3 real uses of AI described

What this episode covers

Jorge Heraud, VP of Autonomy at John Deere, explains how the company leverages its acquisition of Blue River Technology and Spark AI to deploy Level 4 autonomous tractors, using computer vision and machine learning to enable 'See and Spray' and autonomous tillage while managing rural connectivity challenges. RJ He and Ryan McMichael, technology leaders at Zoox, discuss their purpose-built, steering-wheel-free robotaxi recently deployed on public roads in California, detailing a perception stack that integrates cameras, LIDAR, radar, and thermal imaging to ensure safety through redundancy and diversity. The episode contrasts the agricultural application of autonomy, focused on labor relief and precision, with the urban mobility focus of Zoox, highlighting the technical hurdles of sensor fusion, edge cases, and the critical importance of building a robust safety case for public acceptance.

AI in practice

Where this conversation describes AI being used to build or deliver a product. Part of the AI in practice tracker.

Engineering & simulationSpark AI

John Deere uses Spark AI's system to have humans review images from the autonomous tractor when it stops for a potential obstacle, allowing the system to distinguish real obstacles from false positives like dust or shadows and continue driving without interrupting the farmer.

“What we do instead of of bugging the farmer is we send these images to this company that is now part of John Deere, called Spark AI, and they have a really good system where they have 2 humans that independently look at the image … if it's not, if it's a false positive … Then um, then it says, no, there's nothing there. You can continue driving…”

Speaker 3 · Hear it at 22:07

Engineering & simulation

John Deere's autonomous tractor uses machine learning to detect obstacles via cameras and stop conservatively when in doubt, triggering the Spark AI loop described above.

“when in doubt, what we have is our machine learning said to be super conservative. When it doubt, it stops, right? But not all the reasons it stops are really because there's something in front, a real obstacle.”

Speaker 3 · Hear it at 22:07

Engineering & simulation

John Deere acquired Blue River Technology to bring computer vision and machine learning expertise for selective spraying, which is now integrated into their autonomy and perception stacks.

“John Deere acquired Blue River, to bring that technology, the sea and spray technology, uh, into John Deere's portfolio, but also to bring the expertise of perception of computer vision machine learning into John Deere…”

Speaker 3 · Hear it at 12:23

Mentioned

Companies
John Deere, Zoox, Blue River Technology, Spark AI, Starlink, Teladon, Optimus Ride, Cruise
Products
ExactShot, See and Spray, Auto Track
Chips
LIDAR, Radar, Thermal Imaging

Other shows on the same ground, closest first.

  • Over the Air · 18m

    Digital Transformation: From Tractors to Autonomous Agriculture

    Jorge Heraud, VP of Automation and Autonomy at John Deere, explains how the company is leveraging IoT and computer vision to transform agriculture through products like See & Spray and fully autonomous tractors. He details the decision to refine existing herbicide application rather than pursuing unproven technologies like lasers, achieving a 3-5x reduction in chemical usage. Heraud also outlines John Deere's roadmap to a fully autonomous farm by 2030, highlighting the operational advantages of private land environments compared to public roads.

  • IoT For All Podcast · 20m

    Smart Farming: The Role of IoT and AI | John Deere's Gaurav Bansal | Internet of Things Podcast

    Gaurav Bansal, VP of Engineering at Blue River Technology (a subsidiary of John Deere), discusses the integration of IoT, AI, and computer vision in smart farming to address labor shortages and climate challenges. He details the John Deere autonomous tractor, which uses stereo cameras and NVIDIA Jetson GPUs for obstacle detection, and the See & Spray system, which uses real-time computer vision to reduce herbicide usage by up to two-thirds. Bansal also highlights the critical role of connectivity, noting a partnership with Starlink to solve rural coverage gaps, and outlines John Deere's goal of achieving fully autonomous production systems for corn and soybeans by 2030.

  • Robot Talk · 34m

    Episode Eleven – Self-Driving Cars: behind the wheel

    Dr Richard Hawkins of the University of York and Dr Frederic Labrosse of Aberystwyth University discuss the distinct challenges of autonomous vehicle safety, regulation, and perception. Hawkins explains the difficulty of defining safety cases for AI-driven cars in complex urban environments and the UK's new regulations for automated lane keeping systems. Labrosse contrasts on-road challenges, like plastic bag detection, with off-road issues such as terrain variability and LIDAR sensitivity to grass, highlighting applications in farming, archaeology, and surveying.

  • Moore's Lobby · 55m

    Getting Out of the Weeds with John Deere's Revolutionary AI Technology

    Chris Padwick, Director of Computer Vision and Machine Learning at Blue River Technology (a John Deere subsidiary), explains the engineering behind the See & Spray system, which uses computer vision to spot-spray herbicide on weeds. He details the massive compute requirements, noting the system uses NVIDIA Xavier and Orin GPUs to achieve 0.3 petaflops of processing power for real-time image classification at 12 mph. Padwick also discusses the critical role of data collection, the acquisition by John Deere for scaling, and the future of autonomous agricultural machinery.

  • Moore's Lobby · 40m

    From the Ground Up—Zoox Turns a Corner with New AV and Sensor System Designs

    Ryan McMichael, Director of Sensors and Systems at Zoox, discusses the company's purpose-built, driverless robotaxi designed for mobility-as-a-service. He details the sensor suite—including LIDAR, radar, cameras, thermal imaging, and microphones—and the strategy of using diversity and redundancy to ensure safety and uptime. McMichael also reflects on his transition from aerospace instrumentation at Johns Hopkins APL to automotive, highlighting the shift from low-volume, high-stakes systems to high-volume manufacturing with rigorous reliability standards.

The summary, chapters and topics are generated from the episode’s audio by this directory, not written by The Robot Report Podcast. The episode itself belongs to its publisher.