A research team at Seoul National University has developed a robotic AI system called Dynin-Robotics that unifies three capabilities within one model: understanding human instructions, predicting how a scene will change after an action, and generating the physical movements to complete a task. The work is scheduled for presentation at NeurIPS 2026 in December.
Most existing approaches to robot control use separate systems for different functions. Language models interpret instructions, while video models predict future states. Dynin-Robotics handles all three - language, vision, and action - through a shared architecture based on a masked diffusion model, a type of generative AI that learns to reconstruct masked or noisy data.
How the unified model works
The system treats robot trajectories as sequences of discrete tokens representing text, images or video frames, and continuous robot actions. By masking different portions of this sequence during training, the same model learns four separate tasks: generating actions from instructions and current observations, predicting next visual states from actions, generating goal states from instructions, and describing what a robot did based on video of its actions.
This unified training helps the model handle instructions expressed in unfamiliar ways. In simulated tests, the system responded not only to direct commands like "Put the rose in" but also to indirect references such as "Put in the flower that symbolizes love." The ability to predict future scenes also lets the robot evaluate candidate actions before executing them, similar to how a person might mentally rehearse a movement.
The researchers optimized the system for speed. By grouping action information and reusing computations, they achieved up to 29 times faster internal action generation compared to an unoptimized version.
In experiments with a Franka Research 3 robot, the team tested four manipulation tasks: moving fruit, sorting blocks by color, stacking blocks freely, and stacking blocks in a specified color sequence. Dynin-Robotics achieved an average success rate of 78.4% across these conditions.
The researchers plan to extend the technology to manufacturing and logistics tasks such as component handling and assembly. They are also developing applications in surgical assistance through collaboration with Seoul National University College of Medicine.