iBet uBet web content aggregator. Adding the entire web to your favor.
iBet uBet web content aggregator. Adding the entire web to your favor.



Link to original content: https://unpaywall.org/10.1007/S10044-015-0456-4
Face recognition based on Kinect | Pattern Analysis and Applications Skip to main content
Springer Nature Link
Log in

Face recognition based on Kinect

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

In this paper, we present a new algorithm that utilizes low-quality red, green, blue and depth (RGB-D) data from the Kinect sensor for face recognition under challenging conditions. This algorithm extracts multiple features and fuses them at the feature level. A Finer Feature Fusion technique is developed that removes redundant information and retains only the meaningful features for possible maximum class separability. We also introduce a new 3D face database acquired with the Kinect sensor which has released to the research community. This database contains over 5,000 facial images (RGB-D) of 52 individuals under varying pose, expression, illumination and occlusions. Under the first three variations and using only the noisy depth data, the proposed algorithm can achieve 72.5 % recognition rate which is significantly higher than the 41.9 % achieved by the baseline LDA method. Combined with the texture information, 91.3 % recognition rate has achieved under illumination, pose and expression variations. These results suggest the feasibility of low-cost 3D sensors for real-time face recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. We used the MATLAB wrapper that is available at http://www.mathworks.com/matlabcentral/fileexchange/30242.

  2. Since Kinect is not designed specifically for face capture, but for human body from a distance to the sensor, therefore more than 10 bits are used to store the original depth data to allow storage of distance values greater than 1,024 mm. However in our application, after the face is cropped in the preprocessing stage, 256 intensity levels are used to store all the information, as most of the faces only have around 50 mm depth ranges. Therefore, we have no information lost by using an 8-bit intensity image to represent the face depth map.

  3. The Lumix-DMC-FT1 model digital camera.

References

  1. Abate AF, Nappi M, Riccio D, Sabatino G (2007) 2d and 3d face recognition: a survey. Pattern Recogn Lett 28(14):1885–1906

    Article  Google Scholar 

  2. Zhao W, Chellappa R, Phillips P, Rosenfeld A (2003) Face recognition: a literature survey. ACM Comput Surv 35(4):399–458

    Article  Google Scholar 

  3. Bowyer KW, Chang K, Flynn P (2006) A survey of approaches and challenges in 3d and multi-modal 3d + 2d face recognition. Comput Vision Image Underst 101(1):1–15

    Article  Google Scholar 

  4. Wang Y, Liu J, Tang X (2010) Robust 3d face recognition by local shape difference boosting. IEEE Trans Pattern Anal Mach Intell 32(10):1858–1870

    Article  Google Scholar 

  5. Queirolo CC, Silva L, Bellon ORP, Pamplona Segundo M (2010) 3d face recognition using simulated annealing and the surface interpenetration measure. IEEE Trans Pattern Anal Mach Intell 32(2):206–219

    Article  Google Scholar 

  6. Xu C, Li S, Tan T, Quan L (2009) Automatic 3d face recognition from depth and intensity gabor features. Pattern Recogn 42(9):1895–1905

    Article  MATH  Google Scholar 

  7. Mahoor MH, Abdel-Mottaleb M (2009) Face recognition based on 3d ridge images obtained from range data. Pattern Recogn 42(3):445–451

    Article  MATH  Google Scholar 

  8. Sala Llonch R, Kokiopoulou E, Tošić I, Frossard P (2010) 3d face recognition with sparse spherical representations. Pattern Recogn 43(3):824–834

    Article  MATH  Google Scholar 

  9. Phillips PJ, Flynn PJ, Scruggs T, Bowyer KW, Chang J, Hoffman K, Marques J, Min J, Worek W (2005) Overview of the face recognition grand challenge. In: IEEE Int’l Conf. on Computer Vision and Pattern Recognition. pp 947–954

  10. Boehnen C, Flynn P (2005) Accuracy of 3d scanning technologies in a face scanning scenario. In: Int’l Conf. on 3-D Digital Imaging and Modeling. pp 310–317

  11. 3dmdface system. 3dMD. http://www.3dmd.com/3dMDface/

  12. Vivid 910 non-contact 3d digitizer. Konica Minolta Sensing, Inc. http://www.konicaminolta.com

  13. Swissranger sr4000 overview. MESA Imaging. http://www.mesa-imaging.ch/prodview4k.php

  14. Introducing kinect for xbox 360. Microsoft Corp. http://www.xbox.com/en-AU/Kinect

  15. Cignoni P Meshlab. http://meshlab.sourceforge.net/

  16. Khoshelham K (2011) Accuracy analysis of kinect depth data. ISPRS Workshop Laser Scanning 38:1

    Google Scholar 

  17. Openni. OpenNI organization. http://www.openni.org/

  18. Facial action coding system (facs): Manual (1978) Palo Alto: Consulting Psychologists Press

  19. Lucey P, Cohn J, Kanade T, Saragih J, Ambadar Z, Matthews I (2010) The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: IEEE Workshops on Computer Vision and Pattern Recognition. pp 94–101

  20. Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Vision Comput 27(6):803–816

    Article  Google Scholar 

  21. (2005) Automatic facial feature extraction and 3d face modeling using two orthogonal views with application to 3d face recognition. Pattern Recogn 38(12):2549–2563

  22. Technical description of kinect calibration. ROS.org. http://www.ros.org/wiki/kinect_calibration/technical

  23. Zhang Z, Matsushita Y, Ma Y (2011) Camera calibration with lens distortion from low-rank textures. pp 2321–2328

  24. Hansard M, Horaud R, Amat M, Lee S (2011) Projective alignment of range and parallax data. pp 3089–3096

  25. Antoni Liang LLMRF, Liu W, Le V (2014) Accurate facial landmarks detection for frontal faces with extended treestructured models. In: Accepted by 22nd International Conference on Pattern Recognition

  26. Le V, Brandt J, Lin Z, Bourdev L, Huang TS (2012) Interactive facial feature localization. In: Proceedings of the 12th European conference on Computer Vision—vol. part III, ser. ECCV’12. Springer, Berlin, Heidelberg, pp 679–692

  27. Gu L, Kanade T (2006) 3d alignment of face in a single image. pp 1305–1312

  28. Bustard J, Nixon M (2010) 3d morphable model construction for robust ear and face recognition. pp 2582–2589

  29. Li BYL, Mian A, Liu WQ, Krishna A (2013) Using kinect for face recognition under varying poses, expressions, illumination and disguise. In: IEEE WACV. pp 186–192

  30. Yang J, Liu C, Yang J-Y (2010) What kind of color spaces is suitable for color face recognition? Neurocomputing 73(10–12):2140–2146

    Article  Google Scholar 

  31. Ojala T, Pietikainen M, Harwood D (1996) A comparative study of texture measures with classification based on feature distributions. Pattern Recogn 29(1):51–59

    Article  Google Scholar 

  32. Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. In: IEEE Int’l Conf. on Computer Vision and Pattern Recognition. pp I511–I518

  33. Liu C, Wechsler H (2002) Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition. IEEE Trans Image Process 11(4):467–476

    Article  Google Scholar 

  34. Yang J, Liu C, Zhang L (2010) Color space normalization: enhancing the discriminating power of color spaces for face recognition. Pattern Recogn 43(4):1454–1466

    Article  MathSciNet  MATH  Google Scholar 

  35. Thomas M, Kambhamettu C, Kumar S (2008) Face recognition using a color subspace lda approach. In: IEEE Int’l Conf. on Tools with, Artificial Intelligence. pp 231–235

  36. Yang J, Liu C (2008) Color image discriminant models and algorithms for face recognition. IEEE Trans Neural Netw 19(12):2088–2098

    Article  Google Scholar 

  37. Liu C (2008) Learning the uncorrelated, independent, and discriminating color spaces for face recognition. IEEE Trans Inf For Secur 3(2):213–222

    Article  Google Scholar 

  38. Tenllado C, Gómez JI, Setoain J, Mora D, Prieto M (2010) Improving face recognition by combination of natural and gabor faces. Pattern Recogn Lett 31:1453–1460

    Article  Google Scholar 

  39. Ahonen T, Hadid A, Pietikainen M (2004) Face recognition with local binary patterns. Lect Notes Comput Sci 3021:469–481

    Article  MATH  Google Scholar 

  40. Jing X, Liu Q, Lan C, Man J, Li S, Zhang D (2010) Holistic orthogonal analysis of discriminant transforms for color face recognition. In: IEEE Int’l Conf. on Image Processing. pp 3841–3844

  41. Choi J, Ro Y, Plataniotis K (2011) Boosting color feature selection for color face recognition. IEEE Trans Image Process 20(5):1425–1434

    Article  MathSciNet  Google Scholar 

  42. Biometric testing and statistics (2006) National Science & Technology Council (NSTC)

  43. Turk M, Pentland A (1991) Face recognition using eigenfaces. In: IEEE Int’l Conf. on Computer Vision and Pattern Recognition. pp 586–591

  44. Belhumeur P, Hespanha J, Kriegman D (1997) Eigenfaces vs fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

    Article  Google Scholar 

  45. Ohta Y (1985) Knowledge-based interpretation of outdoor natural color scenes. Pitman Publishing Inc, Marshfield

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Curtin University, GPO Box U1987, Perth, WA, 6845, Australia

    Billy Y. L. Li, Wanquan Liu & Aneesh Krishna

  2. The University of Western Australia, 35 Stirling Highway, Crawley, WA, 6009, Australia

    Ajmal S. Mian

Authors
  1. Billy Y. L. Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ajmal S. Mian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wanquan Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Aneesh Krishna
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wanquan Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, B.Y.L., Mian, A.S., Liu, W. et al. Face recognition based on Kinect. Pattern Anal Applic 19, 977–987 (2016). https://doi.org/10.1007/s10044-015-0456-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-015-0456-4

Keywords

Search

Navigation