INCLG: Inpainting for Non-Cleft Lip Generation with a Multi-Task Image Processing Network

Chen, Shuang; Atapour-Abarghouei, Amir; Ho, Edmond S.L.; Shum, Hubert P.H.

Abstract

We present a software that predicts non-cleft facial images for patients with cleft lip, thereby facilitating the understanding, awareness and discussion of cleft lip surgeries. To protect patients’ privacy, we design a software framework using image inpainting, which does not require cleft lip images for training, thereby mitigating the risk of model leakage. We implement a novel multi-task architecture that predicts both the non-cleft facial image and facial landmarks, resulting in better performance as evaluated by surgeons. The software is implemented with PyTorch and is usable with consumer-level color images with a fast prediction speed, enabling effective deployment.

Citation

Chen, S., Atapour-Abarghouei, A., Ho, E. S., & Shum, H. P. (2023). INCLG: Inpainting for Non-Cleft Lip Generation with a Multi-Task Image Processing Network. Software impacts, 17, Article 100517. https://doi.org/10.1016/j.simpa.2023.100517

• Journal Article Type: Article
• Acceptance Date: May 17, 2023
• Online Publication Date: May 22, 2023
• Publication Date: 2023-09
• Deposit Date: May 18, 2023
• Publicly Available Date: Jul 27, 2023
• Journal: Software Impacts
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 17
• Article Number: 100517
• DOI: https://doi.org/10.1016/j.simpa.2023.100517
• Publisher URL: https://www.sciencedirect.com/journal/software-impacts