Face Recognition

Summary

Human identification based on facial traits has made remarkable progress thanks to deep learning-based methods, but there are still challenges, for example related to the effects of occlusions, retrieval from large databases, and multi-modal representations. The recent pandemic has reminded how face recognition systems are deeply affected by facial occlusions due to large sections of the population wearing masks (Randhawa et al., 2022). Retrieving face images from large datasets instead requires learning representations that are compact and balanced to maximize performance (Pidhorskyi et al., 2018)]. The use of multiple modalities, like appearance and depth, instead requires reaching a balance between acquisition cost and capture time even in uncontrolled conditions (Piccirilli et al., 2016).

Related Projects

Face Modeling and Tracking

2 minute read

Active Appearance Models (AAMs) represent facial images with generative models for both shape and appearance of the face. Despite their success, they enjoy l...

References

1. ISVC

   Learning Representations for Masked Facial Recovery Randhawa, Z. A., Patel, S., Adjeroh, D. A., and Doretto, G. In Advances in Visual Computing, 2022. Oral abstract bibTeX arXiv pdf doi

   The pandemic of these very recent years has led to a dramatic increase in people wearing protective masks in public venues. This poses obvious challenges to the pervasive use of face recognition technology that now is suffering a decline in performance. One way to address the problem is to revert to face recovery methods as a preprocessing step. Current approaches to face reconstruction and manipulation leverage the ability to model the face manifold, but tend to be generic. We introduce a method that is specific for the recovery of the face image from an image of the same individual wearing a mask. We do so by designing a specialized GAN inversion method, based on an appropriate set of losses for learning an unmasking encoder. With extensive experiments, we show that the approach is effective at unmasking face images. In addition, we also show that the identity information is preserved sufficiently well to improve face verification performance based on several face recognition benchmark datasets.

   @inproceedings{randhawaPAD22isvc,
     abbr = {ISVC},
     author = {Randhawa, Z. A. and Patel, S. and Adjeroh, D. A. and Doretto, G.},
     booktitle = {Advances in Visual Computing},
     title = {Learning Representations for Masked Facial Recovery},
     year = {2022},
     address = {Cham},
     editor = {Bebis, G. and Li, B. and Yao, A. and Liu, Y. and Duan, Y. and Lau, M. and Khadka, R. and Crisan, A. and Chang, R.},
     pages = {22--35},
     month = oct,
     publisher = {Springer International Publishing},
     bib2html_pubtype = {Conferences},
     doi = {10.1007/978-3-031-20713-6_2},
     arxiv = {2212.14110},
     wwwnote = {Oral}
   }
   

2. ACCV

   Deep supervised hashing with spherical embedding Pidhorskyi, S., Jones, Q., Motiian, S., Adjeroh, D., and Doretto, G. In Proceedings of The Asian Conference on Computer Vision, 2018. abstract bibTeX pdf

   Deep hashing approaches are widely applied to approximate nearest neighbor search for large-scale image retrieval. We propose Spherical Deep Supervised Hashing (SDSH), a new supervised deep hashing approach to learn compact binary codes. The goal of SDSH is to go beyond learning similarity preserving codes, by encouraging them to also be balanced and to maximize the mean average precision. This is enabled by advocating the use of a different relaxation method, allowing the learning of a spherical embedding, which overcomes the challenge of maintaining the learning problem well-posed without the need to add extra binarizing priors. This allows the formulation of a general triplet loss framework, with the introduction of the spring loss for learning balanced codes, and of the ability to learn an embedding quantization that maximizes the mean average precision. Extensive experiments demonstrate that the approach compares favorably with the state-of-the-art while providing significant performance increase at more compact code sizes.

   @inproceedings{pidhorskyiJMAD18accv,
     abbr = {ACCV},
     author = {Pidhorskyi, S. and Jones, Q. and Motiian, S. and Adjeroh, D. and Doretto, G.},
     title = {Deep supervised hashing with spherical embedding},
     booktitle = {Proceedings of The Asian Conference on Computer Vision},
     year = {2018},
     pages = {417--434},
     bib2html_pubtype = {Conferences}
   }
   

3. IEEESJ

   A Mobile Structured Light System for 3D Face Acquisition Piccirilli, M., Doretto, G., Ross, A., and Adjeroh, D. IEEE Sensors Journal, 2016. abstract bibTeX pdf

   A mobile sensor based on fringe projection techniques is developed with the goal of acquiring face 3D and color with a smartphone device. The system consists of a portable pico-projector and an Android-based smartphone. The data acquisition, pattern generation. and reconstruction of the final 3D point cloud are all driven by the smartphone. We present results on the root-mean-square error (RMSE) of the sensor and on 3D face matching.

   @article{piccirilliDRA2015sj,
     abbr = {IEEESJ},
     author = {Piccirilli, M. and Doretto, G. and Ross, A. and Adjeroh, D.},
     title = {{A Mobile Structured Light System for 3D Face Acquisition}},
     journal = {IEEE Sensors Journal},
     year = {2016},
     volume = {16},
     number = {7},
     pages = {1854--1855},
     owner = {doretto},
     timestamp = {2015.09.16},
     bib2html_pubtype = {Journals}
   }