Real Time 3d Reconstruction Of Dynamic Scenes

Real world digitization is a very important but challenging topic in Computer Graphics and Vision. One prime challenge is how to achieve real-time 3D reconstruction, which has a high demand in various application fields: Virtual Reality, Augmented Reality, Autonomous Driving, Health Care and so on. The most challenging problem of real-time 3D reconstruction is dynamic scenes reconstruction such as human bodies or non-human objects in motions. Existing work in this field has largely centered around fusion based framework with RGBD(epth) images. Prior knowledge such as predefined skeleton or pre-scanned template is usually employed to reduce the solution space and generate more robust results. However, prior knowledge has big limitation that it is very hard to extract the prior knowledge for everything in the world. Another issue we are facing is the lack of solutions for topology change problems. In this dissertation, we propose a unified framework to reconstruct both human body and non-human objects without any prior knowledge. With the help of learned segmentation, the 3D reconstruction of human and non-human objects is enabled under some highly-occluded and challenging motions. We continue by developing methods to solve reconstruction of dynamic scenes with topology changes. We also propose several techniques to accelerate our algorithms to achieve real-time performance. Finally, we show how our proposed methods outperform the state-of-the-art methods in several challenging cases.