Oscar Messelt and William Reit presents their master’s thesis Enhancing Deep Learning-Based 3D Face Reconstructions with Consumer-Grade Depth Data Thursday 5 June at 15:15 in MH:309A

Abstract:
Deep learning models have significantly improved the advancement of monocular 3D face reconstruction models. However, such models often struggle to capture person-specific high-frequency details. Often, because they are trained on large, images in the wild, datasets of thousands of different individuals. To address this issue, this report proposes a hybrid reconstruction pipeline that enhances DECA produced 3D face models by incorporating RGB-D data captured with the consumer grade iPhone TrueDepth camera. While the original goal was to modify the DECA code to train on RGB-D data, stability issues with the original code led us to focus on post-processing the DECA generated 3D facial model with depth data. The final pipeline uses DECA output, synchronized RGB-D, face segmentation, landmark detection, and ICP. This method allows for correcting 3D facial models to better reflect individual characteristics. The corrected reconstructions show an improved detail correspondence to the target person, having converged towards the mean shape of several depth scans.

Examiner: 
Viktor Larsson, Centre for Mathematical Sciences, Lund University

Supervisors:
Kalle Åström, Centre for Mathematical Sciences, Lund University
Andreas Rodman, Lingotion AB