Cutting Edge: AI-Powered Landscaping with Electric Sheep

Engineering & simulationNVIDIA Isaac Sim

The team used NVIDIA Isaac Sim to create procedural lawns and parks for training reinforcement learning policies in simulation before fine-tuning them in the real world.

“with our collaboration and video, we use Isaac Sims to create procedural lawns and parks, um, so we have, basically, you can think of it, like, park world where it's just, like, a bunch of different random grasses are generated. And then you have robots training, doing policy learning in STIM, um, and learning how to be robust to like a wide variety of parks and sites.”

Engineering & simulation

The company developed a dense transformer model that predicts common robotics representations, such as bird's eye view maps and agent poses, to create a large world model that feeds into policy training.

“So we have a dense transformer model that's predicting a lot of the common representations you might see in robotics, like a bird's eye view map, a post where it's been, um, the semantics of the world. And then what's nice about that architecture is you have the interpretability.”

Engineering & simulation9 months of R&D

The engineering team pivoted from classical teach-and-repeat robotics to an ML-first approach, removing the ability to teach the robot and relying on ML trained in simulation to handle coverage planning autonomously.

“we decided 2023 was the year to make machine learning the primary agent for our robotic fleet, this wasn't an easy choice, and definitely had significant challenges, but after 9 months of R&D, the results have blown us away.”