3D Reconstructing Semantically Meaningful Baggage Models From Synthetically Generated 3D Point Clouds

Ark, van, Rosalie

3D Reconstructing Semantically Meaningful Baggage Models From Synthetically Generated 3D Point Clouds

Files

Ark, van, R. s-1006233-MSc-MKI92-Thesis-2023.pdf (22.74 MB)

Authors

Ark, van, Rosalie

Issue Date

2023-03-13

Language

en

URI

https://theses.ubn.ru.nl/handle/123456789/16389

Abstract

Vanderlande’s current ULD stacking algorithms are primarily based on using heuristics such as “biggest suitcase first and smallest later”. Although this is an effective method for simple nondeformable shapes, it does not translate well to more complex domains, such as the “baggage” domain. Therefore, this thesis proposed a pre-processing pipeline using the physics simulation environment Isaac Sim as an aid for the stacking challenge. The pipeline specifically explored how suitcases can be reconstructed to semantically meaningful and low-complexity meshes from 3D point clouds. Suitcases were 3D scanned, from which synthetic 3D point cloud data was gathered. The semantic 3D part segmentation model PointNet++ was trained on this data and provided these point clouds with a semantic meaning according to the relevant attributes. Subsequently, the EMS algorithm was implemented to reconstruct the individual attributes to a simplistic mesh based on superquadrics, after which the reconstructed individual attributes were put together again, resulting in a simplified suitcase 3D model. The results showed that although this pipeline has a lot of potential, it is not yet ready to be implemented in the real world. The biggest limitation for the PointNet++ model appeared to be the lack of data and its variety, particularly for the handles class, which PointNet++ struggled with the most. Generating more data and augmenting it could overcome this issue. The 3D reconstruction results showed that the EMS algorithm only provides robust reconstructions for point clouds with a complete surface. Therefore, a better approach might be to base the choice in reconstruction method on the type of attribute.