An AI capable of reconstructing 3D scenes from photos in just a few seconds


You may also like this partner content (after the club)

At one time, it seemed incredible sober to discover that the Polarod can almost instantly develop a picture that has just been taken. Today, the system capable of creating 3D scenes from just a few pictures in a few seconds has just been presented by NVIDIA.

The sober effect company has developed a method which allows a sober artificial cleverness to convert sober 2D pictures in just a few seconds into a single 3D scene. With great fidelity, believe the demonstration video. We can indeed discover a photograph tribute to Andy Warhol con, a reference to the one where we saw him take a Polarod photograph, put in three dimensions. A camera then turns around the scene, showing that we can look at the details from every angle.

Known as sober inverse rendering, this process uses artificial intelligence (AI) to make a sober approximation of real-world light behavior, allowing sober to reconstruct a 3D scene from a handful of 2D images taken under different perspectives , explains NVIDIA in a content blog page devoted to this creativity.

NVIDIA has applied this approach to a technology that has received a great deal of attention since its inception: the neural radiance areas , or, sober shorter, NeRFs. This is a method to generate new views of things that scenes. The NeRF takes the set of input images from a global scene and completes it by linking different sub-scenes. Sober short, it can be terrible that the AI ​​fills in the blanks.

Collecting data to power a NeRF is a bit like being a photographer on the red carpet trying soberly to capture a celebrity outfit from all sides, NVDIA explains soberly fawn picture. The neural network requires a few dozen images taken from several opportunities around the scene, as well as the place of the camera for each of them .

One speed multiplies by

Many Impressive results could be obtained from using this technology. Some research even focuses on the faithful reconstruction of entire cities. In the present case, the scale is certainly more tiny. What makes the novelty of what NVIDIA offers is rather the speed. According to the company, the whole process, from learning the pictures by the AI ​​to the final rendering, takes only a few seconds: all this would therefore be up to times faster than what AIs are usually able to provide in this area.

This new method, built on the existing one, the capital has been baptized Quick NeRF . This, according to the company, has recently published research results on this subject which, reduces the rendering of several orders of magnitude. It is based on a method called multi-resolution hash grid encoding, is optimized to run efficiently on NVIDIA GPUs which. Sober using a new input coding method, researchers can achieve high quality results using a very low neural network that works fast.

Uses for this invention are particularly envisaged in the fields of autonomous cars and robotics. This algorithm could indeed allow these devices to better visualize and understand their environment, a potentially sufficient speed for the adaptation to be achievable. Instant NeRF could also be used in architecture and entertainment to quickly generate digital representations of real environments that creators can modify and expand , adds NVIDIA.