On-board AI for robotic operations

2024-10-02
2 min read.
On-board AI software enables robots to quickly map a scene and identify the items they need to complete a given set of tasks.

MIT engineers have developed a method that enables robots to make intuitive, task-relevant decisions in complex environments in the real world. The applications of the new method range from factory and domestic robots to search and rescue operations.

The new method is called Clio, like the Greek muse of history, for its ability to identify and remember only the things that really matter for a given task. For example, a quadruped robot (Spot by Boston Dynamics) running Clio on-board in real-time identified and mapped only those parts of an office building that related to the robot’s tasks (such as retrieving a dog toy while ignoring piles of office supplies), allowing the robot to grasp the objects of interest.

MIT’s Clio runs in real-time to map task-relevant objects in a robot’s surroundings, allowing the bot (Boston Dynamic’s quadruped robot Spot, pictured) carry out a natural language task  (Credit: MIT).
MIT’s Clio runs in real-time to map task-relevant objects in a robot’s surroundings, allowing the bot (Boston Dynamic’s quadruped robot Spot, pictured) carry out a natural language task (Credit: MIT).

The Clio system is described in a research paper published in Robotics and Automation Letters. The final manuscript of the paper is available from MIT, and a preprint is published in arXiv.

With Clio, a robot is given a list of tasks described in natural language and, based on those tasks, it analyzes its surroundings to identify and remember only what is relevant to the given tasks.

“Search and rescue is the motivating application for this work, but Clio can also power domestic robots and robots working on a factory floor alongside humans,” says lead researcher Luca Carlone in a MIT press release. “It’s really about helping the robot understand the environment and what it has to remember in order to carry out its mission.”

Combining computer vision and LLMs

On-board Artificial Intelligence (AI) for robots that must navigate and operate in complex environments is important and critical. Such systems would be useful not only for search and rescue operations and the other applications mentioned in this study, but also for all applications that require robots to operate as autonomously as possible, such as law enforcement operations, military operations, and operations in deep space.

Clio makes use of both computer vision and large language models (LLMs), and can be seen as an advance toward multimodal AI.

#AIApplications

#AIinRobotics

#RobotLearning

#RobotLocomotion

#RobotManipulation

#RobotMotionPlanning



Related Articles


Comments on this article

Before posting or replying to a comment, please review it carefully to avoid any errors. Reason: you are not able to edit or delete your comment on Mindplex, because every interaction is tied to our reputation system. Thanks!

Mindplex

Mindplex is an AI company, a decentralized media platform, a global brain experiment, and a community dedicated to the rapidly unfolding future. Our platform empowers our community to share and discuss futurist content while showcasing AI and blockchain tools that enhance the media experience. Join us and shape the future of digital media!

ABOUT US

FAQ

CONTACT

Editors

© 2025 MindPlex. All rights reserved