Thesis of Xiangnan Yin

GAN based image synthesizing and it's application in face recognition

Defense date: 06/06/2022

Advisor: Liming Chen


Generative Adversarial Networks (GANs) is recently one of the most popular research directions in the domain of the generative model of computer vision. Benefiting from the adversarial training paradigm, GANs could gradually generate data close to the real data distribution, which provides the possibility of generating photo-realistic images. 
Recently, GANs are combined with the conditional auto-encoder and widely used in face image generation, e.g., facial attributes editing and face pose synthesizing. However, whether the synthesized data could improve the face recognition accuracy remains to be a question. 

In our works, we studied the state-of-the-art face image generation algorithms, and propose an algorithm of face pose synthesizing.  Based on the observation that face images of different poses share a large number of pixels, a Pixel Attention Sampling module is designed. The module could select the pixels from the source image, and use them to construct the target image, which could largely preserve the details and the style of the source image. Using this module, we convert the face pose synthesizing problem to face image inpainting problem, which could widely extend the training data from paired face images to unpaired face images. We also use pre-detected 3D facial landmarks to represent the face pose, which is more flexible and more precise comparing to the one-hot pose label or the 2D facial landmarks. The proposed algorithm outperforms the state-of-the-art algorithms both in the image quality and the identity preserving ability.

In the future, we will combine GAN based algorithms and 3D morphable models to explore the 3D face generation problem. With the generated 3D face data, we could get face images in arbitrary poses and illuminations, which could further improve the face recognition accuracy.

Mr Ben Amor BoulbabaProfesseur(e)Inception Institute of Artificial Intelligence à Abu Dhabi, EAURapporteur(e)
Mr Marcel SébastienChercheurInstitut de Recherche Idiap, SuisseRapporteur(e)
Mme Caplier AliceProfesseur(e)Grenoble-INP
Mme Dantcheva AntitzaChercheurEquipe STARS - INRIA Méditerranée - Sophia Antipolis
Mr Gentric StéphaneProfesseur(e) associé(e)Responsable d'unité de recherche chez Idemia et professeur asocié au LCTI, Télécom ParisTech
Mr Chen LimingProfesseur(e)LIRIS CNRS UMR5205 - ECL LyonDirecteur(trice) de thèse
Mr Huang Di Professeur(e)Université de Beihang, ChineCo-directeur (trice)