Ambulatory Human Gait Phase Detection Using Wearable Inertial Sensors and Hidden Markov Model
Abstract
:1. Introduction
2. Related Works
3. Analysis of Hidden Markov Model
3.1. Analysis of HMM Theory
- (1)
- N: The number of states contained in a model is usually determined before the model is built. Suppose that the state at time t is , then .
- (2)
- M: The number of observation values corresponding to each state in the model (when the output observation value is discrete value), if the observation value of the model at time t is , then .
- (3)
- A: The state transition probability matrix of the model is . If the state of the model at time t is , the transition probability can be expressed as , and the state at time is , where , T is the length of the model output observation value and . Meanwhile, the state transition matrix A satisfies .
- (4)
- : If the state of the model at a certain time is and the output observation value is , the relationship between the state and the observation value can be expressed as , and the observation value probability matrix needs to meet .
- (5)
- : The probability of occurrence of each state in HMM model at the first time, , when the initial state is , can be expressed as . The initial state probability needs to satisfy .
3.2. Implementation of EM Algorithm
4. Materials and Methods
4.1. Gait Data Collection
4.1.1. Data Acquisition Hardware Platform
4.1.2. Software Platform
4.2. Gait Data Preprocessing
4.3. Window Segmentation for Gait Data
5. Experimental Results
5.1. Experimental Data Source
5.1.1. Data Collection Object
5.1.2. Gait Data Collection
5.2. Model Performance Evaluation Metrics
- TP: The model identifies a positive sample as a positive sample.
- FN: The model identifies a positive sample as a negative sample.
- FP: The model identifies a negative sample as a positive sample.
- TN: The model identifies a negative sample as a negative sample.
5.3. Analysis of Results Based on Hidden Markov Models and Improved Models
5.3.1. Recognition Results of HMM with Gaussian Distribution
5.3.2. Recognition Results of HMM with Gaussian Mixed Distribution
5.4. Identification Results after Model Parameter Adaptation
5.4.1. Performance Analysis of Gaussian HMM after Parameter Adaptation
5.4.2. Performance Analysis of Gaussian Mixed HMM with Adaptive Parameters
5.5. Comparison with Other Algorithms
5.6. Analysis of Human Balance Ability by Gait Parameters
5.7. Experimental Conclusions
6. Discussions and Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
Abbreviations
MEMS | Micro-electro-mechanical Sensor |
BSN | Body Sensor Network |
IMU | Inertial Measurement Unit |
ZUPT | Zero Velocity Updating |
HMM | Hidden Markov Model |
References
- Barth, J.; Oberndorfer, C.; Pasluosta, C.; Schülein, S.; Gassner, H.; Reinfelder, S.; Kugler, P.; Schuldhaus, D.; Winkler, J.; Klucken, J.; et al. Stride segmentation during free walk movements using multi-dimensional subsequence dynamic time warping on inertial sensor data. Sensors 2015, 15, 6419–6440. [Google Scholar] [CrossRef]
- Bhargava, N.; Taro, N.; Amanda, R.; Jerry, T.; Jerry, L.; Tam, N.; Steven, Z.; Donald, L. Real-Time Classification of Patients with Balance Disorders vs. Normal Subjects Using a Low-Cost Small Wireless Wearable Gait Sensor. Biosensors 2016, 6, 58. [Google Scholar]
- Bao, S.D.; Meng, X.L.; Xiao, W.; Zhang, Z.Q. Fusion of inertial/magnetic sensor measurements and map information for pedestrian tracking. Sensors 2017, 17, 340. [Google Scholar] [CrossRef] [PubMed]
- Majumder, S.; Mondal, T.; Deen, M.J. A Simple, Low-cost and Efficient Gait Analyzer for Wearable Healthcare Applications. IEEE Sens. J. 2018, 19, 2320–2329. [Google Scholar] [CrossRef]
- Lahmiri, S. Gait Nonlinear Patterns Related to Parkinson’s Disease and Age. IEEE Trans. Instrum. Meas. 2019, 68, 2545–2551. [Google Scholar] [CrossRef]
- Chiang, C.Y.; Chen, K.H.; Liu, K.C.; Hsu, S.; Chan, C.T. Data Collection and Analysis Using Wearable Sensors for Monitoring Knee Range of Motion after Total Knee Arthroplasty. Sensors 2017, 17, 418. [Google Scholar] [CrossRef] [PubMed]
- Hayati, H.; Mahdavi, F.; Eager, D. Analysis of agile canine gait characteristics using accelerometry. Sensors 2019, 19, 4379. [Google Scholar] [CrossRef] [Green Version]
- Wang, H.; Li, L.; Chen, H.; Li, Y.; Qiu, S. Motion Recognition for Smart Sports Based on Wearable Inertial Sensors. In Proceedings of the 14th EAI International Conference on BodyNets 2019, Florence, Italy, 2–3 October 2019; pp. 114–124. [Google Scholar]
- Wang, Z.; Liu, R.; Zhao, H.; Qiu, S.; Shi, X.; Wang, J.; Li, J. Motion Analysis of Deadlift for Trainers with Different Levels based on Body Sensor Network. IEEE Trans. Instrum. Meas. 2021, 71, 1–12. [Google Scholar]
- Wang, Y.; Wang, H.; Xuan, J.; Leung, D.Y.C. Powering future body sensor network systems: A review of power sources. Biosens. Bioelectron. 2020, 166, 112410. [Google Scholar] [CrossRef]
- Qiu, S.; Wang, Z.; Zhao, H.; Hu, H. Using Distributed Wearable Sensors to Measure and Evaluate Human Lower Limb Motions. IEEE Trans. Instrum. Meas. 2016, 65, 939–950. [Google Scholar] [CrossRef] [Green Version]
- Gravina, R.; Li, Q. Emotion-relevant activity recognition based on smart cushion using multi-sensor fusion. Inf. Fusion 2019, 48, 1–10. [Google Scholar] [CrossRef]
- Li, Q.; Raffaele, G.; Qiu, S.; Wang, Z.; Zang, W.; Li, Y. Group Walking Recognition Based on Smartphone Sensors. In Proceedings of the 14th EAI International Conference on BodyNets 2019, Florence, Italy, 2–3 October 2019; Volume 1, pp. 91–102. [Google Scholar]
- Ma, C.; Li, W.; Cao, J.; Du, J.; Li, Q.; Gravina, R. Adaptive sliding window based activity recognition for assisted livings. Inf. Fusion 2020, 53, 55–65. [Google Scholar] [CrossRef]
- Li, J.; Wang, Z.; Qiu, S.; Zhao, H. Using Body Sensor Network to Measure the Effect of Rehabilitation Therapy on Improvement of Lower Limb Motor Function in Children with Spastic Diplegia. IEEE Trans. Instrum. Meas. 2020, 69, 9215–9227. [Google Scholar] [CrossRef]
- Choe, N.; Zhao, H.; Qiu, S.; So, Y. A sensor-to-segment calibration method for motion capture system based on low cost MIMU. Meas. J. Int. Meas. Confed. 2019, 131, 490–500. [Google Scholar] [CrossRef]
- Liu, L.; Wang, Z.; Qiu, S. Driving Behavior Tracking and Recognition Based on Multisensors Data Fusion. IEEE Sens. J. 2020, 20, 10811–10823. [Google Scholar] [CrossRef]
- Gravina, R.; Ma, C.; Pace, P.; Aloi, G.; Russo, W.; Li, W.; Fortino, G. Cloud-based Activity-aaService cyber–physical framework for human activity monitoring in mobility. Future Gener. Comput. Syst. 2017, 75, 158–171. [Google Scholar] [CrossRef]
- Qiu, S.; Wang, Z.; Zhao, H.; Liu, L.; Li, J.; Jiang, Y.; Fortino, G. Body Sensor Network based Robust Gait Analysis: Toward Clinical and at Home Use. IEEE Sens. J. 2019, 19, 8393–8401. [Google Scholar] [CrossRef]
- Liu, L.; Wang, H.; Qiu, S.; Zhang, Y.; Hao, Z. Paddle Stroke Analysis for Kayakers Using Wearable. Sensors 2021, 21, 914. [Google Scholar] [CrossRef] [PubMed]
- Fortino, G.; Di Fatta, G.; Pathan, M.; Vasilakos, A.V. Cloud-assisted body area networks: State-of-the-art and future challenges. Wirel. Netw. 2014, 20, 1925–1938. [Google Scholar] [CrossRef]
- Li, Q.; Gravina, R.; Li, Y.; Alsamhi, S.H.; Sun, F.; Fortino, G. Multi-user activity recognition: Challenges and opportunities. Inf. Fusion 2020, 63, 121–135. [Google Scholar] [CrossRef]
- Mannini, A.; Sabatini, A.M. Machine learning methods for classifying human physical activity from on-body accelerometers. Sensors 2010, 10, 1154–1175. [Google Scholar] [CrossRef] [Green Version]
- Nickel, C.; Busch, C. Classifying accelerometer data via hidden Markov models to authenticate people by the way they walk. IEEE Aerosp. Electron. Syst. Mag. 2013, 28, 29–35. [Google Scholar] [CrossRef]
- Huang, J.; Yi, Z.; Wang, X.; Wu, H. Gait recognition system based on (2D)2 PCA and HMM. In Proceedings of the Eighth International Conference on Digital Image Processing, Chengu, China, 20–22 May 2016. [Google Scholar]
- Cavallo, F.; Sabatini, A.M.; Genovese, V. A step toward GPS/INS personal navigation systems: Real-time assessment of gait by foot inertial sensing. In Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, Edmonton, AB, Canada, 2–6 August 2005; pp. 1187–1191. [Google Scholar]
- Qiu, S.; Wang, Z.; Zhao, H. Heterogeneous data fusion for three-dimensional gait analysis using wearable MARG sensors. Int. J. Comput. Sci. Eng. 2017, 14, 222–233. [Google Scholar] [CrossRef] [Green Version]
- Wang, Z.; Qiu, S.; Cao, Z.; Jiang, M. Quantitative assessment of dual gait analysis based on inertial sensors with body sensor network. Sens. Rev. 2013, 33, 48–56. [Google Scholar] [CrossRef]
- Stalin, M.; Bennett, C.L. Development of an Insole System for Real-time Capture of Ground Reaction Forces in Lower-limb Amputees. In Proceedings of the 2013 29th Southern Biomedical Engineering Conference, Seville, Spain, 25–28 September 2013; pp. 137–138. [Google Scholar]
- Schepers, H.M.; Roetenberg, D.; Veltink, P.H. Ambulatory human motion tracking by fusion of inertial and magnetic sensing with adaptive actuation. Med. Biol. Eng. Comput. 2010, 48, 27–37. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Qiu, S.; Wang, H.; Li, J.; Zhao, H.; Wang, Z.; Wang, J.; Wang, Q.; Plettemeier, D.; Bärhold, M.; Bauer, T.; et al. Towards Wearable-Inertial-Sensor-Based Gait Posture Evaluation for Subjects with Unbalanced Gaits. Sensors 2020, 20, 1193. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Mikos, V.; Heng, C.H.; Tay, A.; Yen, S.C.; Chia, N.S.Y.; Koh, K.M.L.; Tan, D.M.L.; Au, W.L. A Wearable, Patient-Adaptive Freezing of Gait Detection System for Biofeedback Cueing in Parkinson’s Disease. IEEE Trans. Biomed. Circuits Syst. 2019, 13, 503–515. [Google Scholar] [CrossRef]
- Sun, F.; Zang, W.; Gravina, R.; Fortino, G.; Li, Y. Gait-based identification for elderly users in wearable healthcare systems. Inf. Fusion 2020, 53, 134–144. [Google Scholar] [CrossRef]
- Ahmad, Z.; Khan, N. Human Action Recognition Using Deep Multilevel Multimodal (M2) Fusion of Depth and Inertial Sensors. IEEE Sens. J. 2020, 20, 1445–1455. [Google Scholar] [CrossRef] [Green Version]
- Anwary, A.R.; Yu, H.; Vassallo, M. Optimal Foot Location for Placing Wearable IMU Sensors and Automatic Feature Extraction for Gait Analysis. IEEE Sens. J. 2018, 18, 2555–2567. [Google Scholar] [CrossRef]
- Zhao, H.; Wang, Z.; Qiu, S. Adaptive gait detection based on foot-mounted inertial sensors and multi-sensor fusion. Inf. Fusion 2019, 52, 157–166. [Google Scholar] [CrossRef]
- Brzostowski, K. Toward the Unaided Estimation of Human Walking Speed Based on Sparse Modeling. IEEE Trans. Instrum. Meas. 2018, 67, 1389–1398. [Google Scholar] [CrossRef]
- Qiu, S.; Wang, Z.; Zhao, H.; Liu, L.; Jiang, Y. Using Body-Worn Sensors for Preliminary Rehabilitation Assessment in Stroke Victims with Gait Impairment. IEEE Access 2018, 6, 31249–31258. [Google Scholar] [CrossRef]
- An, W.W.; Ting, K.H.; Au, I.P.; Zhang, J.H.; Chan, Z.Y.; Davis, I.S.; So, W.K.; Chan, R.H.; Cheung, R.T. Neurophysiological Correlates of Gait Retraining With Real-Time Visual and Auditory Feedback. IEEE Trans. Neural Syst. Rehabil. Eng. 2019, 27, 1341–1349. [Google Scholar] [CrossRef] [PubMed]
- Ahmed, M.; Naude, J.; Birkholtz, F.; Glatt, V.; Tetsworth, K. Gait & Posture The relationship between gait and functional outcomes in patients treated with circular external fi xation for malunited tibial fractures. Gait Posture 2019, 68, 569–574. [Google Scholar]
- Qiu, S.; Yang, Y.; Hou, J.; Ji, R. Ambulatory estimation of 3D walking trajectory and knee joint angle using MARG Sensors. In Proceedings of the Fourth International Conference on Innovative Computing Technology (INTECH), Luton, UK, 13–15 August 2014; pp. 191–196. [Google Scholar]
- Teknomo, K.; Estuar, M.R. Visualizing Gait Patterns of Able bodied Individuals and Transtibial Amputees with the Use of Accelerometry in Smart Phones. Rev. Colomb. Estad. 2014, 37, 471–488. [Google Scholar] [CrossRef]
- Wang, J.; Wang, Z.; Qiu, S.; Xu, J.; Zhao, H.; Fortino, G.; Habib, M. A selection framework of sensor combination feature subset for human motion phase segmentation. Inf. Fusion 2021, 70, 1–11. [Google Scholar] [CrossRef]
- Zhao, H.; Wang, Z.; Qiu, S.; Shen, Y.; Zhang, L.; Tang, K. Heading Drift Reduction for Foot-Mounted Inertial Navigation System via Multi-Sensor Fusion and Dual-Gait Analysis. IEEE Sens. J. 2019, 19, 8514–8521. [Google Scholar] [CrossRef]
- Noor, M.H.M.; Salcic, Z.; Wang, I.K. Adaptive sliding window segmentation for physical activity recognition using a single tri-axial accelerometer. Pervasive Mob. Comput. 2016, 38, 41–59. [Google Scholar] [CrossRef]
- Caderby, T.; Yiou, E.; Peyrot, N.; Bonazzi, B.; Dalleau, G. Detection of swing heel-off event in gait initiation using force-plate data. Gait Posture 2013, 37, 463–466. [Google Scholar] [CrossRef] [Green Version]
- Farah, J.D.; Baddour, N.; Lemaire, E.D. Gait phase detection from thigh kinematics using machine learning techniques. In Proceedings of the 2017 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Rochester, MN, USA, 7–10 May 2017. [Google Scholar]
- Wang, T.; Wang, Z.; Zhang, D.; Tao, G.U.; Hongbo, N.I.; Jia, J.; Zhou, X.; Jing, L.V. Recognizing Parkinsonian Gait Pattern by Exploiting Fine-Grained Movement Function Features. ACM Trans. Intell. Syst. Technol. 2016, 8, 1–22. [Google Scholar] [CrossRef]
- Qiu, S.; Liu, L.; Zhao, H.; Wang, Z.; Jiang, Y. MEMS Inertial Sensors Based Gait Analysis for Rehabilitation Assessment via Multi-Sensor Fusion. Micromachines 2018, 9, 442. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Liu, Y.; Yue, L.; Hou, J. Gait recognition based on MEMS accelerometer. In Proceedings of the IEEE 10th International Conference on Signal Processing Proceedings, Beijing, China, 24–28 October 2010. [Google Scholar]
- Leal-Junior, A.G.; Frizera, A.; Avellar, L.M.; Marques, C.; Pontes, M.J. Polymer Optical Fiber for In-Shoe Monitoring of Ground Reaction Forces during the Gait. IEEE Sens. J. 2018, 18, 2362–2368. [Google Scholar] [CrossRef]
- Martinez-Hernandez, U.; Mahmood, I.; Dehghani-Sanij, A.A. Simultaneous Bayesian Recognition of Locomotion and Gait Phases with Wearable Sensors. IEEE Sens. J. 2018, 18, 1282–1290. [Google Scholar] [CrossRef] [Green Version]
- Qiu, S.; Wang, Z.; Zhao, H.; Qin, K.; Li, Z.; Hu, H. Inertial/magnetic sensors based pedestrian dead reckoning by means of multi-sensor fusion. Inf. Fusion 2018, 39, 108–119. [Google Scholar] [CrossRef] [Green Version]
- Meng, X.; Zhang, Z.Q.; Wu, J.K.; Wong, W.C.; Yu, H. Self-contained pedestrian tracking during normal walking using an inertial/magnetic sensor module. IEEE Trans. Biomed. Eng. 2014, 61, 892–899. [Google Scholar] [CrossRef] [Green Version]
- Young, S.S.; Sangkyung, P. Pedestrian inertial navigation with gait phase detection assisted zero velocity updating. In Proceedings of the ICARA 2009—4th International Conference on Autonomous Robots and Agents, Wellington, New Zealand, 10–12 February 2009; pp. 336–341. [Google Scholar]
- Suh, Y.S. A smoother for attitude and position estimation using inertial sensors with zero velocity intervals. IEEE Sens. J. 2012, 12, 1255–1262. [Google Scholar] [CrossRef]
- Feliz, R.; Zalama, E.; Garcia-Bermejo, J.G. Pedestrian tracking using inertial sensors. J. Phys. Agents 2009, 3, 35–43. [Google Scholar]
- Skog, I.; Händel, P.; Nilsson, J.O.; Rantakokko, J. Zero-velocity detection—An algorithm evaluation. IEEE Trans. Biomed. Eng. 2010, 57, 2657–2666. [Google Scholar] [CrossRef]
Index | Formulation | Definition |
---|---|---|
Accuracy rate (A) | The percentage of samples correctly identified as a percentage of the samples used for testing indicates the overall identification rate of the system. | |
Recalling rate (R) | The ratio of correctly identified positive samples in all positive, indicating the identification rate ofindividual states of the HMM model. | |
Precision ratio (P) | The proportion of correctly identified positive samples in all positive samples, showing the effect of sample distribution on recognition rate. | |
F1 value | The reconciled mean of P and R was combined to evaluate P and R. | |
Sensitivity (TPR) | Sensitivity is the same as recalling rate. | |
False accuracy (TPR) | TPR is predicted to be the ratio of the positive sample to the negative sample. | |
ROC Curve | Not applicable | The larger the area enclosed by the curve, the higher the classification rate. |
Parameters | Mean Value | Variance |
---|---|---|
HMM | [−2.6518 3.7528 −1.0578 −0.1528 ] | [3.9164 2.4664 0.7880 0.0146] |
MLLR | [−2.4168 3.6096 −1.3194 −0.2117] | [3.0880 2.2895 0.8294 0.0184] |
MAP | [−2.3472 3.5249 −1.5249 −0.2398] | [2.7750 2.2851 0.7171 0.0192] |
Performance Index | Without Parameter Adaptive | MLLR | MAP |
---|---|---|---|
Accuracy rate (A) | |||
Precision ratio (P) | 0.8876, 0.9303, 0.9234, 0.9146 | 0.8885, 0.9383, 0.9268, 0.8869 | 0.8938, 0.9386, 0.9196, 0.9111 |
Recalling rate (R) | |||
Precision ratio (P) | 0.8910, 0.9578, 0.8368, 0.9251 | 0.8986, 0.9453, 0.7917, 0.9469 | 0.9176, 0.9497, 0.8142, 0.9386 |
F1 value | 0.8893, 0.9439, 0.8780, 0.9198 | 0.8935, 0.9418, 0.8539, 0.9160 | 0.9056, 0.9441, 0.8637, 0.9247 |
Sensitivity (TPR) | 0.0308, 0.0424, 0.0127, 0.0307 | 0.0309, 0.0363, 0.0113, 0.0437 | 0.0300, 0.0364, 0.0129, 0.0327 |
Reference Value of LLL | Reference Value of RLL | Recognition Value of LLL | Recognition Value of RLL | |
---|---|---|---|---|
Swing time (s) | ||||
Standing time (s) | ||||
Gait cycle (s) |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Liu, L.; Wang, H.; Li, H.; Liu, J.; Qiu, S.; Zhao, H.; Guo, X. Ambulatory Human Gait Phase Detection Using Wearable Inertial Sensors and Hidden Markov Model. Sensors 2021, 21, 1347. https://doi.org/10.3390/s21041347
Liu L, Wang H, Li H, Liu J, Qiu S, Zhao H, Guo X. Ambulatory Human Gait Phase Detection Using Wearable Inertial Sensors and Hidden Markov Model. Sensors. 2021; 21(4):1347. https://doi.org/10.3390/s21041347
Chicago/Turabian StyleLiu, Long, Huihui Wang, Haorui Li, Jiayi Liu, Sen Qiu, Hongyu Zhao, and Xiangyang Guo. 2021. "Ambulatory Human Gait Phase Detection Using Wearable Inertial Sensors and Hidden Markov Model" Sensors 21, no. 4: 1347. https://doi.org/10.3390/s21041347
APA StyleLiu, L., Wang, H., Li, H., Liu, J., Qiu, S., Zhao, H., & Guo, X. (2021). Ambulatory Human Gait Phase Detection Using Wearable Inertial Sensors and Hidden Markov Model. Sensors, 21(4), 1347. https://doi.org/10.3390/s21041347