Mask-FPAN: Semi-supervised face parsing in the wild with de-occlusion and UV GAN

Publikation: Bidrag til tidsskriftTidsskriftartikelForskningfagfællebedømt


  • Mask-FPAN

    Indsendt manuskript, 13,3 MB, PDF-dokument

The field of fine-grained semantic segmentation for a person's face and head, which includes identifying facial parts and head components, has made significant progress in recent years. However, this task remains challenging due to the difficulty of considering ambiguous occlusions and large pose variations. To address these difficulties, we propose a new framework called Mask-FPAN. Our framework includes a de-occlusion module that learns to parse occluded faces in a semi-supervised manner, taking into account face landmark localization, face occlusion estimations, and detected head poses. Additionally, we improve the robustness of 2D face parsing by combining a 3D morphable face model with the UV GAN. We also introduce two new datasets, named FaceOccMask-HQ and CelebAMaskOcc-HQ, to aid in face parsing work. Our proposed Mask-FPAN framework successfully addresses the challenge of face parsing in the wild and achieves significant performance improvements, with a mIoU increase from 0.7353 to 0.9013 compared to the current state-of-the-art on challenging face datasets.

TidsskriftComputers and Graphics (Pergamon)
Sider (fra-til)185-193
Antal sider9
StatusUdgivet - 2023

Bibliografisk note

Funding Information:
This work was supported by Villum Fonden through the Deep Learning and Remote Sensing for Unlocking Global Ecosystem Resource Dynamics (DeReEco) project (grant number: 34306 ).

Publisher Copyright:
© 2023 Elsevier Ltd

ID: 366981605