Prediction of Patient Demographics using 3D Craniofacial Scans and Multi-view CNNs

Publikation: Bidrag til bog/antologi/rapportKonferencebidrag i proceedingsForskningfagfællebedømt

  • Umaer Hanif
  • Rasmus R. Paulsen
  • Eileen B. Leary
  • Emmanuel Mignot
  • Jennum, Poul
  • Helge B.D. Sorensen

3D data is becoming increasingly popular and accessible for computer vision tasks. A popular format for 3D data is the mesh format, which can depict a 3D surface accurately and cost-effectively by connecting points in the (x, y, z) plane, known as vertices, into triangles that can be combined to approximate geometrical surfaces. However, mesh objects are not suitable for standard deep learning techniques due to their non-euclidean structure. We present an algorithm which predicts the sex, age, and body mass index of a subject based on a 3D scan of their face and neck. This algorithm relies on an automatic pre-processing technique, which renders and captures the 3D scan from eight different angles around the x-axis in the form of 2D images and depth maps. Subsequently, the generated data is used to train three convolutional neural networks, each with a ResNet18 architecture, to learn a mapping between the set of 16 images per subject (eight 2D images and eight depth maps from different angles) and their demographics. For age and body mass index, we achieved a mean absolute error of 7.77 years and 4.04 kg/m2 on the respective test sets, while Pearson correlation coefficients of 0.76 and 0.80 were obtained, respectively. The prediction of sex yielded an accuracy of 93%. The developed framework serves as a proof of concept for prediction of more clinically relevant variables based on 3D craniofacial scans stored in mesh objects.

OriginalsprogEngelsk
Titel42nd Annual International Conferences of the IEEE Engineering in Medicine and Biology Society : Enabling Innovative Technologies for Global Healthcare, EMBC 2020
Antal sider4
ForlagIEEE
Publikationsdato2020
Sider1950-1953
Artikelnummer9176333
ISBN (Elektronisk)9781728119908
DOI
StatusUdgivet - 2020
Begivenhed42nd Annual International Conferences of the IEEE Engineering in Medicine and Biology Society, EMBC 2020 - Montreal, Canada
Varighed: 20 jul. 202024 jul. 2020

Konference

Konference42nd Annual International Conferences of the IEEE Engineering in Medicine and Biology Society, EMBC 2020
LandCanada
ByMontreal
Periode20/07/202024/07/2020
NavnProceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
ISSN2375-7477

Bibliografisk note

Publisher Copyright:
© 2020 IEEE.

ID: 262894297