Augmented reality 3D maps and social distance monitoring systems, all you need is a smartphone

It is now possible to use the images captured by a single camera, such as that of a smartphone, to create complex 3D maps in real-time. These maps can then be implemented in different applications such as those for augmented reality or the monitoring of social distancing.

The applications developed at the University of Bologna and presented today at Dubai Expo are part of the Internet of Things project that involves all the four universities of the Emilia-Romagna region. The Computer Vision Lab of the Department of Computer Science and Engineering participated for the University of Bologna.

Researchers will show how to employ deep-learning techniques applied to the recordings of one or more cameras to retrieve detailed information on the 3D structure of an environment.

“This research topic is of great interest for different fields of applications such as autonomous driving, augmented reality, IOT systems, robotics, and video surveillance,” explains Professor Stefano Mattoccia. “Systems based on deep-learning models now represent the state of the art. Our research group is very active in this field, especially with regards to unsupervised learning models that greatly simplify the training of neural networks.”

Researchers have developed neural networks able to infer in real-time the 3D structure of an environment by taking the data from low power consumption devices with only one camera (monocular vision systems) such as those of smartphones.

Building on these results, two new applications will be presented at Dubai Expo. One allows to create detailed 3D maps in real-time to increase the consistency of augmented reality with the real environment.

In fact, by processing a single image through a neural network, the system can recreate its 3D structure. This can then be used in many applications such as the integration of virtual objects within the real environment where they can interact consistently.

The second application presented is a real-time social distancing monitoring system. The underlying principle is again the creation of 3D maps from monocular vision systems, i.e. systems with only one camera that can be used in any environment.

Once the camera has been installed, the software will automatically generate a 3D map that is then resized to determine the absolute distance between people. This allows real-time monitoring of social distancing between people in a certain area for an effective risk assessment: from low or no risk to moderate risk to the warning of serious violations of protocols on social distancing.