NVIDIA Research has released a revolutionary new AI model called Neuralangelo. Using neural networks, the cutting-edge model can recreate 3D sceneries and objects from 2D video footage. 

In the same way that Michelangelo sculpted lifelike images from marble blocks, Neuralangelo gives life to still images by turning them into elaborate 3D constructions. This technology allows creative people from various areas to bring lifelike virtual copies into their design programmes. Neuralangelo will likely revolutionise how we create and interact with virtual objects, from art and video game development to robotics and industrial digital twins.

Neuralangelo

Neuralangelo creates 3D models with realistic textures and features, much like Michelangelo's lifelike sculptures. Then, artists, game developers, roboticists, and industrial digital twins can take these 3D models and use them in their projects.

Compared to other methods, Neuralangelo excels at converting the textures of complicated materials like roof shingles, glass, and smooth marble from 2D films to 3D assets. The outstanding quality of its 3D reconstructions makes it much simpler for programmers and artists to create realistic virtual things from smartphone footage quickly.

NVIDIA researchers demonstrated the model's ability to reconstruct famous works of art like Michelangelo's David and everyday items like a flatbed truck. The park on NVIDIA's Bay Area headquarters served as an excellent example of Neuralangelo's ability to recreate both the interior and exterior of a building.

3D Model

Early AI reconstructions of 3D scenes needed help capturing textures with large colour variations, uniform colours, and repetitive patterns. Neuralangelo uses the NVIDIA Instant NeRF technology, which is based on instant neural graphics primitives, to aid in capturing these nuances.

The model uses a 2D video of an object or scene filmed from several angles to select multiple frames that capture diverse viewpoints, much like an artist might do when looking at a subject from different perspectives to get a feel for its depth, size, and shape. After figuring out where the camera was in each frame, Neuralangelo's AI generates a basic 3D model of the scene, much like a sculptor carving away at a block of marble. Just as a sculptor meticulously carves stone to imitate the texture of the fabric or a human body, the model optimises the render to sharpen the details. The end output is a 3D item or expansive scene suitable for use in VR, digital twin, or robotics research and development.

Computer vision

Neuralangelo is one of approximately 30 experiments presented by NVIDIA Research at the Conference on Computer Vision and Pattern Recognition (CVPR) from June 18 to 22 in Vancouver. These articles cover various issues, such as posture estimation, 3D reconstruction, and video creation. The presentation of Neuralangelo at this prominent conference emphasises its potential to revolutionise the field of computer vision.

Conclusion

The Neuralangelo AI model developed by NVIDIA Research has ushered in a new era of 3D scene reconstruction. Neuralangelo enables creative people across sectors by transforming 2D films into complex, lifelike virtual creations. Everything from buildings and games to robots and digital duplicates is within reach. Neuralangelo blurs the lines between the real and the virtual with incredible realism and user-friendliness.

Image source: Unsplash