Researchers at the Shanghai Jiao Tong University have published a new technique for coordinating a robot’s whole body – arms and legs – to improve how it handles real-world objects.

It may seem more obvious to use the legs for one purpose (moving about), and the arm for another (performing the task), but the research showed that better results are achieved if the legs get involved in the manipulation.

The robot used in the research was a quadruped with a single arm, and it was tested picking up 14 different objects at various heights.

The robot “could operate fully autonomously using only visual input and its proprioception”, rather than using electronic rangefinders or other scanning methodologies. This is comparable to a person picking things up using mostly visual cues and proprioception. The researchers call their method ‘Visual Whole-Body Control’ (VBC), because it is controlled visually and uses the whole body as one piece.

The key to their technique is providing simple goals that govern the whole body together: “Our low-level goal-reaching policy controls the quadruped robot and the mounted arm simultaneously. For the quadruped robot, our low-level policy outputs the target joint angles for all 12 joints of the robot”.

This has an obvious similarity with human movement: you’ll find it easier to crack an egg or hammer a nail if you position your legs right. Surely it’s a good idea to teach robots the same tricks of proper technique that we teach people.

Citation: Liu, Minghuan ; Chen, Zixuan ; Cheng, Xuxin ; Ji, Yandong ; Yang, Ruihan ; Wang, Xiaolong. Visual Whole-Body Control for Legged Loco-Manipulation. eprint (2024). https://arxiv.org/abs/2403.16967 (open access)

Let us know your thoughts! Sign up for a Mindplex account now, join our Telegram, or follow us on Twitter.