Creating 3D models for uses like virtual reality, movies, and engineering can be hard and slow. Generative Artificial Intelligence (AI) models can make realistic 2D images from text but not 3D shapes.

A new technique called Score Distillation uses 2D image models to make 3D shapes, but these shapes often look blurry or cartoonish.

MIT researchers found out why Score Distillation’s 3D shapes are not as sharp as 2D images. They developed a simple fix to make the 3D shapes look better. Instead of retraining expensive AI models, their method uses the models as they are.

The researchers looked at examples like a rotating robotic bee and a strawberry to show how their new technique works with text-based AI. Their approach makes 3D shapes that look just as good or better than other methods without extra training or complex fixes.

The researchers have described the methods and results of this study in a paper published in arXiv.

Applications to VR and engineering

They discovered that Score Distillation struggles because it adds random noise in a way that doesn’t match the 2D image generation. They solved this by approximating the noise in a smarter way, making the 3D shapes sharper. They also improved the image quality and adjusted model settings for better results.

This technique doesn’t require retraining the AI, which saves time and money. However, it still uses the same AI, so if that AI has flaws, those will show in the 3D shapes too. The team is now thinking about how to solve the math behind this technique even better and how these ideas might help with editing images.

“In the long run, our work can help facilitate the process to be a co-pilot for designers, making it easier to create more realistic 3D shapes,” says researcher Artem Lukoianov in an MIT press release.

This seems a boon for virtual reality (VR) and engineering design. For VR, higher-quality 3D models mean more immersive and realistic environments, enhancing user experience and interaction. In engineering, detailed, accurate 3D models allow for better prototyping, simulation, and visualization of designs before physical production, which can reduce costs and errors in manufacturing.

Let us know your thoughts! Sign up for a Mindplex account now, join our Telegram, or follow us on Twitter.