Learning from Unstructured Geometry: Quadric Error Metrics for Clinical Mesh Deep Learning

Nistelrooij, van, Niels

Learning from Unstructured Geometry: Quadric Error Metrics for Clinical Mesh Deep Learning

Files

Nistelrooij, s-4713648-2021.pdf (62.21 MB)

Authors

Nistelrooij, van, Niels

Issue Date

2021-08-06

Language

en

URI

https://theses.ubn.ru.nl/handle/123456789/16102

Abstract

Orthognathic surgery cuts and reshapes the jaw bones. The procedure is performed to improve jaw function, which entails correcting malocclusion or asymmetries. The surgery results in signi cant changes to the patient's facial pro le, making it viable for improving aesthetics as well. Presenting the expected changes to a patient's facial pro le before they commit to surgery can alleviate pre-operative anxiety for informed decision making. Furthermore, the surgeon can virtually plan the jaw displacements better by taking facial aesthetics into account, which increases patient satisfaction and reduces re-surgery. This thesis focusses on creating a solution to predicting 3D facial pro les. For that purpose, a novel extension to the Minkowski Engine has been developed that performs deep learning on discrete representations of 3D photogrammetry meshes. The method discretizes meshes with a minimal loss of information into a sparse grid. The grid cells contain quadric error metrics (QEMs) which are based on an arbitrary number of triangle vertex attributes. A QEMs representation based on the mesh's vertex coordinates was predicted using deep learning. This was subsequently transformed back into a mesh by nding the optimal vertex position in each grid cell and connecting these grid cell vertices with triangles. Extensive experiments were conducted comparing the standard Minkowski Engine to the QEMs extension on classi cation, semantic segmentation, and mesh generation tasks. Furthermore, two clinical tasks were implemented. The rst task predicted whether a facial pro le requires orthognathic surgery, which can be used as a pre-screening test. The main task predicted the facial pro le mesh itself after orthognathic surgery and gave control over the realized jaw displacements. Clear improvements over the standard Minkowski Engine were observed for the QEMs extension, which shows its general utility for mesh deep learning. The orthognathic classi er achieved an accuracy of 91%, which is in the realm of clinical applicability. The predicted post-operative facial pro le meshes showed clear jaw displacements and presented an average error of 1.77 mm compared to the post-operative target meshes. Despite the potential of the novel approach, the predictions were less accurate than the state of the art. The potential of the QEMs extension for 3D clinical data has been demonstrated beyond any doubt. However, more research is necessary to improve the e ectiveness for clinical applicability. Ultimately, the implemented framework of tasks will serve as the foundation on which many other 3D clinical tasks will be solved.