In a new article published in Optica , researchers succeeded in creating three-dimensional videos with a camera containing only one pixel. The secret is an AI capable of reconstructing the scene, image by image, from the time of flight of the photons.
A single pixel would be enough to capture three-dimensional images . This is the subject of a new article published in the journal Optica , which brings together researchers from the United Kingdom, Italy and the Netherlands. Rather than using spatial information to take a photo, they reconstructed the scene using the temporal information of the photons measured by a single point sensor .
This new 3D camera is based on the same principle as the time of flight (ToF) sensors that equip some smartphones . A laser emits brief pulses of light, and a sensor measures the time it takes for photons to reach an object and bounce back to the camera. This makes it possible to deduce the distance, and therefore to obtain information in three dimensions on the scene. The difference here is that the researchers are relying on a single point sensor to measure the travel time of photons, rather than hundreds of thousands of pixels.
An AI reconstructs the scene from the temporal graph of photons
The data received is a simple graph of time. The researchers then used artificial intelligence (AI) to reconstruct the scene in three dimensions. They trained a neural network algorithm by providing it with photos of scenes in the lab, along with data simultaneously measured by the single point sensor. By comparing the two, the AI was able to learn to reconstruct the images purely from the temporal data.
The researchers managed to create three-dimensional videos with around ten frames per second, but the equipment used could produce thousands of frames per second. Their technique works with pulsed laser light , but also with radar waves, and could be adapted to any technology that measures the echo of pulses. With more training, the AI could piece together other scenes and be used to improve navigation in self- driving cars , or even monitor the breathing of patients in the hospital, the researchers said.