Robots can now figure out their own body structure and movements by watching themselves with a camera. A new study from Columbia Engineering researchers shows this clearly. With this skill, robots plan their actions better and even handle damage to their parts.

The robots use raw video to understand their bodies, much like humans learning to dance by watching a mirror. The goal is simple. The researchers want robots to adapt to damage and learn new tasks without humans always stepping in.

The study is published in Nature Machine Intelligence.

Usually, robots start learning in simulations, which are virtual spaces. After practicing there, they move to the real world. Good simulators make this switch easier. However, building these simulators takes a lot of effort from skilled engineers. In this study, the researchers found a shortcut. They taught a robot to make its own simulator just by watching its motions on camera. This saves time and lets the simulator grow with the robot.

The researchers used a single 2D camera to help the robot model its 3D shape. Deep neural networks powered this process. The neural networks turned 2D video into 3D motion clues, helping the robot adapt. The system also spotted changes, like a bent arm, and adjusted the robot’s movements to keep it going.

Robots could imagine their future actions

The researchers imagine a robot vacuum with a bent arm after hitting a chair. Instead of stopping, it watches itself, tweaks its moves, and keeps cleaning. In a car factory, a robot arm might get misaligned. Rather than pausing work, it adjusts itself and resumes welding, saving time and money.

Robots must handle critical tasks, like in factories or hospitals. Humans can’t always fix them. Self-modeling helps robots care for themselves and stay useful. Now, with one camera and a short video, robots gain what researchers call “Kinematic Self-Awareness.” Robots could imagine their future actions, unlocking endless possibilities.

Let us know your thoughts! Sign up for a Mindplex account now, join our Telegram, or follow us on Twitter.