back Back

Robots learn better by also listening

Jul. 04, 2024.
1 min. read Interactions

Stanford researchers find improved robot training by adding audio

About the writer

Amara Angelica

198.01421 MPXR

Audio engineer/electronic musician

Adding a microphone to teach robotic arm to better execute a task (credit: Zeyi Liu)

For robots to move into homes, they’ll need to learn to listen, suggests MIT Technology Review.

“Researchers at the Robotics and Embodied AI Lab at Stanford University have built a system for collecting audio data, consisting of a GoPro camera and a gripper with a microphone.

“Thus far, robots have been training on videos that are muted,” says Zeyi Liu, a PhD student at Stanford and lead author of the study. “But there is so much helpful data in audio.”

The results, published in a paper on arXiv: “When using vision alone in the dice test, the robot could tell 27% of the time if there were dice in the cup, but that rose to 94% when sound was included.”

Citation: Zeyi Liu et al. ManiWAV: Learning Robot Manipulation from
In-the-Wild Audio-Visual Data. arXiv (open access)

Let us know your thoughts! Sign up for a Mindplex account now, join our Telegram, or follow us on Twitter

Comment on this content


0 thoughts on “Robots learn better by also listening




💯 💘 😍 🎉 👏
🟨 😴 😡 🤮 💩

Here is where you pick your favorite article of the month. An article that collected the highest number of picks is dubbed "People's Choice". Our editors have their pick, and so do you. Read some of our other articles before you decide and click this button; you can only select one article every month.

People's Choice