Abstract
Stroke can cause significant damage to neurons, resulting in various sequelae that negatively impact the patient’s ability to perform essential daily activities such as chewing, swallowing, and verbal communication. Therefore, it is important for patients with such difficulties to undergo a treatment process and be monitored during its execution to assess the improvement of their health condition. The use of computerized tools and algorithms that can quickly and affordably detect such sequelae proves helpful in aiding the patient’s recovery. Due to the death of internal brain cells, a stroke often leads to facial paralysis, resulting in certain asymmetry between the two sides of the face. This paper focuses on analyzing this asymmetry using a deep learning method without relying on handcrafted calculations, introducing the Facial Point Graphs (FPG) model, a novel approach that excels in learning geometric information and effectively handling variations beyond the scope of manual calculations. FPG allows the model to effectively detect orofacial impairment caused by a stroke using video data. The experimental findings on the Toronto Neuroface dataset revealed the proposed approach surpassed state-of-the-art results, promising substantial advancements in this domain.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
For the BLOW subtask, there were 15 participants, consisting of 8 stroke patients and 7 healthy controls (HC).
- 2.
\(C =[2^{-5}, 2^{-3}, .. 2^{15}]\), and \(\gamma = [2^{-15}, 2^{-13}, ..., 2^{3}]\).
- 3.
Note: The experimental evaluation of FPG was conducted using PyTorch Geometric [9] on a GPU. An important point to mention is that, up to the current paper, the implementation of Graph Neural Networks (GNNs) using scatter operation on a GPU introduces non-deterministic behavior.
References
Baltrusaitis, T., Zadeh, A., Lim, Y.C., Morency, L.P.: Openface 2.0: facial behavior analysis toolkit. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 59–66. IEEE (2018)
Bandini, A., Green, J.R., Richburg, B., Yunusova, Y.: Automatic detection of orofacial impairment in stroke. In: Interspeech, pp. 1711–1715 (2018)
Bandini, A., et al.: A new dataset for facial motion analysis in individuals with neurological disorders. IEEE J. Biomed. Health Inform. 25(4), 1111–1119 (2020)
Bulat, A., Tzimiropoulos, G.: How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks). In: International Conference on Computer Vision (2017)
Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20, 273–297 (1995)
Delaunay, B., et al.: Sur la sphere vide. Izv. Akad. Nauk SSSR, Otdelenie Matematicheskii i Estestvennyka Nauk 7(793-800), 1–2 (1934)
Dhall, A., Goecke, R., Lucey, S., Gedeon, T., et al.: Collecting large, richly annotated facial-expression databases from movies. IEEE Multimedia 19(3), 34 (2012)
Dong, X., Yan, Y., Ouyang, W., Yang, Y.: Style aggregated network for facial landmark detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 379–388 (2018)
Fey, M., Lenssen, J.E.: Fast graph representation learning with pytorch geometric. arXiv preprint arXiv:1903.02428 (2019)
Fix, E., Hodges, J.L.: Discriminatory analysis. nonparametric discrimination: Consistency properties. Inter. Stat. Review/Revue Internationale de Statistique 57(3), 238–247 (1989)
Gomes, N.B., Yoshida, A., Roder, M., de Oliveira, G.C., Papa, J.P.: Facial point graphs for amyotrophic lateral sclerosis identification. arXiv preprint arXiv:2307.12159 (2023)
Greene, J.J., et al.: The spectrum of facial palsy: the meei facial palsy photo and video standard set. Laryngoscope 130(1), 32–37 (2020)
Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. Image Vis. Comput. 28(5), 807–813 (2010)
Guarin, D.L., et al.: Toward an automatic system for computer-aided assessment in facial palsy. Facial Plastic Surgery Aesthetic Med. 22(1), 42–49 (2020)
Haykin, S.: Neural networks: a comprehensive foundation. Prentice Hall PTR (1994)
Ho, T.K.: Random decision forests. In: Proceedings of 3rd International Conference on Document Analysis and Recognition, vol. 1, pp. 278–282. IEEE (1995)
Kaewmahanin, W., et al.: Automatic facial asymmetry analysis for elderly stroke detection by using cosine similarity. In: 2022 19th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), pp. 1–4. IEEE (2022)
Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867–1874 (2014)
Kim, H.S., Kim, S.Y., Kim, Y.H., Park, K.S.: A smartphone-based automatic diagnosis system for facial nerve palsy. Sensors 15(10), 26756–26768 (2015)
King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)
Lapchak, P.A., Zhang, J.H.: The high cost of stroke and stroke cytoprotection research. Transl. Stroke Res. 8, 307–317 (2017)
Lou, J., Yu, H., Wang, F.Y.: A review on automated facial nerve function assessment from visual face capture. IEEE Trans. Neural Syst. Rehabil. Eng. 28(2), 488–497 (2019)
Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended cohn-kanade dataset (ck+): a complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-workshops, pp. 94–101. IEEE (2010)
Mullen, M., Loomis, C., et al.: Differentiating facial weakness caused by bell’s palsy vs. acute stroke. JEMS (2014)
Newell, A., Yang, K., Deng, J.: Stacked hourglass networks for human pose estimation. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 483–499. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_29
Ngoc, Q.T., Lee, S., Song, B.C.: Facial landmark-based emotion recognition via directed graph neural network. Electronics 9(5), 764 (2020)
Pantic, M., Valstar, M., Rademaker, R., Maat, L.: Web-based database for facial expression analysis. In: 2005 IEEE International Conference on Multimedia and Expo, pp. 5–pp. IEEE (2005)
Parra-Dominguez, G.S., Sanchez-Yanez, R.E., Garcia-Capulin, C.H.: Facial paralysis detection on images using key point analysis. Appl. Sci. 11(5), 2435 (2021)
Pecundo, A.M., Abu, P.A., Alampay, R.: Amyotrophic lateral sclerosis and post-stroke orofacial impairment video-based multi-class classification. In: Proceedings of the 2022 5th Artificial Intelligence and Cloud Computing Conference, pp. 150–157 (2022)
Samsudin, W.W., Sundaraj, K.: Image processing on facial paralysis for facial rehabilitation system: A review. In: 2012 IEEE International Conference on Control System, Computing and Engineering, pp. 259–263. IEEE (2012)
Scarselli, F., Gori, M., Tsoi, A.C., Hagenbuchner, M., Monfardini, G.: The graph neural network model. IEEE Trans. Neural Netw. 20(1), 61–80 (2008)
Schimmel, M., Ono, T., Lam, O., Müller, F.: Oro-facial impairment in stroke patients. J. Oral Rehabil. 44(4), 313–326 (2017)
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Lio, P., Bengio, Y.: Graph attention networks. arXiv preprint arXiv:1710.10903 (2017)
Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 532–539 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 Springer Nature Switzerland AG
About this paper
Cite this paper
Gomes, N.B., Yoshida, A., de Oliveira, G.C., Roder, M., Papa, J.P. (2024). Facial Point Graphs for Stroke Identification. In: Vasconcelos, V., Domingues, I., Paredes, S. (eds) Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications. CIARP 2023. Lecture Notes in Computer Science, vol 14469. Springer, Cham. https://doi.org/10.1007/978-3-031-49018-7_49
Download citation
DOI: https://doi.org/10.1007/978-3-031-49018-7_49
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-49017-0
Online ISBN: 978-3-031-49018-7
eBook Packages: Computer ScienceComputer Science (R0)
