Extending and Applying Single-View 3D Model Reconstruction Work on Complex Real World Objects

Naus, Jordy

Extending and Applying Single-View 3D Model Reconstruction Work on Complex Real World Objects

Files

Naus, J. s-4722426-MSc-MKI92-Thesis-2022.pdf (7.41 MB)

Authors

Naus, Jordy

Issue Date

2022-07-06

Language

en

URI

https://theses.ubn.ru.nl/handle/123456789/16343

Abstract

Single-View 3D Reconstruction is a challenging task in the field of computer vision with the goal to create a 3D model from one single image of an object. With the rapid development of machine learning, and in particular deep learning, recent work on Single-View 3D Reconstruction has seen great improvements. Although the aforementioned techniques have resulted in better performance on this task, it is important to realize that the majority of the work is focused on solving this task using the same datasets that contain relatively simple objects. This thesis aims to explore the field of Single-View 3D Reconstruction and to extend and apply a state-of-the-art framework to more detailed and complex objects, namely eyewear. Of specific interest is how to cope with the challenges introduced by the lenses of the glasses specifically, since state-of-the-art 3D reconstruction cannot predict lenses as separate meshes with a different texture. In this work, we show that the problem with the lenses can be tackled by omitting them in the renders and the labels during training the framework. Furthermore, we will show that the current state-of-the-art can be extended to more complex objects by combining state-of-the-art with a loss function that better takes the surface of predicted 3D meshes into account. We also investigate other factors that influence the performance of machine learning on Single-View 3D Reconstruction. We show that the 3D reconstruction method used is robust to the number of vertices used during training, with performance significantly decreasing only if fewer than 100 vertices are used as ground truth. We also show that the method used to sample these vertices influences the performance on Single-View 3D Reconstruction. Finally, we show that using a domain-specific mesh to deform into the target 3D model does not result in better-looking 3D models.