{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,13]],"date-time":"2024-09-13T15:18:40Z","timestamp":1726240720904},"reference-count":45,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2019,8,20]],"date-time":"2019-08-20T00:00:00Z","timestamp":1566259200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U1333119"],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Defense Industrial Technology Development Program","award":["JCKY2013605B002","GCZX-2015-05","KYCX18_0309"]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"publisher","award":["56XBC18206, 56XBA18201"],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"To realize an early warning of unbalanced workload in the aircraft cockpit, it is required to monitor the pilot\u2019s real-time workload condition. For the purpose of building the mapping relationship from physiological and flight data to workload, a multi-source data fusion model is proposed based on a fuzzy neural network, mainly structured using a principal components extraction layer, fuzzification layer, fuzzy rules matching layer, and normalization layer. Aiming at the high coupling characteristic variables contributing to workload, principal component analysis reconstructs the feature data by reducing its dimension. Considering the uncertainty for a single variable to reflect overall workload, a fuzzy membership function and fuzzy control rules are defined to abstract the inference process. An error feedforward algorithm based on gradient descent is utilized for parameter learning. Convergence speed and accuracy can be adjusted by controlling the gradient descent rate and error tolerance threshold. Combined with takeoff and initial climbing tasks of a Boeing 737\u2013800 aircraft, crucial performance indicators\u2014including pitch angle, heading, and airspeed\u2014as well as physiological indicators\u2014including electrocardiogram (ECG), respiration, and eye movements\u2014were featured. The mapping relationship between multi-source data and the comprehensive workload level synthesized using the NASA task load index was established. Experimental results revealed that the predicted workload corresponding to different flight phases and difficulty levels showed clear distinctions, thereby proving the validity of data fusion.<\/jats:p>","DOI":"10.3390\/s19163629","type":"journal-article","created":{"date-parts":[[2019,8,21]],"date-time":"2019-08-21T15:19:06Z","timestamp":1566400746000},"page":"3629","source":"Crossref","is-referenced-by-count":5,"title":["Adaptive Neuro-Fuzzy Fusion of Multi-Sensor Data for Monitoring a Pilot\u2019s Workload Condition"],"prefix":"10.3390","volume":"19","author":[{"given":"Xia","family":"Zhang","sequence":"first","affiliation":[{"name":"College of Civil Aviation, Nanjing University of Aeronautics & Astronautics, Nanjing 211106, China"}]},{"given":"Youchao","family":"Sun","sequence":"additional","affiliation":[{"name":"College of Civil Aviation, Nanjing University of Aeronautics & Astronautics, Nanjing 211106, China"}]},{"given":"Zhifan","family":"Qiu","sequence":"additional","affiliation":[{"name":"Shanghai Aircraft Design & Research Institute, Commercial Aircraft Corporation of China, Ltd., Shanghai 201210, China"}]},{"given":"Junping","family":"Bao","sequence":"additional","affiliation":[{"name":"Shanghai Aircraft Design & Research Institute, Commercial Aircraft Corporation of China, Ltd., Shanghai 201210, China"}]},{"given":"Yanjun","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Yangzhou University, Yangzhou 225127, China"}]}],"member":"1968","published-online":{"date-parts":[[2019,8,20]]},"reference":[{"key":"ref_1","unstructured":"Cooper, G.E., and Harper, R.P. (1969). The Use of Pilot Rating in the Evaluation of Aircraft Handling Qualities, National Aeronautics and Space Administration."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1016\/S0166-4115(08)62386-9","article-title":"Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research","volume":"Volume 52","author":"Hancock","year":"1988","journal-title":"Advances in Psychology"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"904","DOI":"10.1177\/154193120605000909","article-title":"NASA-task load index (NASA-TLX) 20 years later","volume":"Volume 50","author":"Hart","year":"2006","journal-title":"Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/S0166-4115(08)62387-0","article-title":"The subjective workload assessment technique: A scaling procedure for measuring mental workload","volume":"Volume 52","author":"Hancock","year":"1988","journal-title":"Advances in Psychology"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1518\/155534307X255627","article-title":"A random glance at the flight deck: Pilots\u2019 scanning strategies and the real-time assessment of mental workload","volume":"1","author":"Nocera","year":"2007","journal-title":"J. Cogn. Eng. Decis. Mak."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.neubiorev.2012.10.003","article-title":"Measuring neurophysiological signals in aircraft pilots and car drivers for the assessment of mental workload, fatigue and drowsiness","volume":"44","author":"Borghini","year":"2014","journal-title":"Neurosci. Biobehav. Rev."},{"key":"ref_7","unstructured":"Hoepf, M., Middendorf, M., Epling, S., and Galster, S. (2015). Physiological Indicators of Workload in a Remotely Piloted Aircraft Simulation, Air Force Research Lab Wright-Patterson AFB OH Human Performance Wing (711th)-Human Effectiveness Directorate\/Applied Neuroscience Branch."},{"key":"ref_8","unstructured":"Hoepf, M., Middendorf, M., Epling, S., and Galster, S. (2015). Evaluation of Physiologically-Based Artificial Neural Network Models to Detect Operator Workload in Remotely Piloted Aircraft Operations, Air Force Research Laboratory."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.apergo.2018.01.005","article-title":"Measuring mental workload and physiological reactions in marine pilots: Building bridges towards redlines of performance","volume":"69","author":"Orlandi","year":"2018","journal-title":"Appl. Ergon."},{"key":"ref_10","first-page":"1","article-title":"Investigating the relationship between mental state (workload and affect) and physiology in a control room setting (ship bridge simulator)","volume":"21","author":"Wulvik","year":"2019","journal-title":"Cogn. Technol. Work"},{"key":"ref_11","unstructured":"Ayaz, H., and Dehais, F. (2019). Computational models for near-real-time performance predictions based on physiological measures of workload. Neuroergonomics, Elsevier Academic Press."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.ergon.2015.01.010","article-title":"Effect of time pressure and target uncertainty on human operator performance and workload for autonomous unmanned aerial system","volume":"51","author":"Liu","year":"2016","journal-title":"Int. J. Ind. Erg."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.ergon.2016.09.003","article-title":"Workload profiles: A continuous measure of mental workload","volume":"63","author":"Rusnock","year":"2018","journal-title":"Int. J. Ind. Ergon."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1080\/00140139.2018.1545054","article-title":"Learning rate and subjective mental workload in five truck driving tasks","volume":"62","author":"Chi","year":"2019","journal-title":"Ergonomics"},{"key":"ref_15","first-page":"115","article-title":"Estimation for driver fatigue with phase locking value","volume":"14","author":"Kong","year":"2012","journal-title":"Int. J. Bioelectromagn."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"548602","DOI":"10.1155\/2015\/548602","article-title":"A system of driving fatigue detection based on machine vision and its application on smart device","volume":"2015","author":"Kong","year":"2015","journal-title":"J. Sens."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"24191","DOI":"10.3390\/s150924191","article-title":"A self-adaptive dynamic recognition model for fatigue driving based on multi-source information and two levels of fusion","volume":"15","author":"Sun","year":"2015","journal-title":"Sensors"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1016\/j.ijpsycho.2015.10.004","article-title":"Towards an effective cross-task mental workload recognition model using electroencephalography based on feature selection and support vector machine regression","volume":"98","author":"Ke","year":"2015","journal-title":"Int. J. Psychophysiol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1016\/j.eswa.2018.06.027","article-title":"Brain-computer interface for workload estimation: Assessment of mental efforts in learning processes","volume":"112","author":"Zammouri","year":"2018","journal-title":"Expert Syst. Appl."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"McDonald, A.D., Ferris, T.K., and Wiener, T.A. (2019). Classification of driver distraction: A comprehensive analysis of feature generation, machine learning, and input measures. Hum. Factors.","DOI":"10.1177\/0018720819856454"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1016\/j.artmed.2018.06.005","article-title":"Detecting mental fatigue from eye-tracking data gathered while watching video: Evaluation in younger and older adults","volume":"91","author":"Yamada","year":"2018","journal-title":"Artif. Intell. Med."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1007\/s11571-019-09523-2","article-title":"Mental fatigue level detection based on event related and visual evoked potentials features fusion in virtual indoor environment","volume":"13","author":"Lamti","year":"2019","journal-title":"Cogn. Neurodynamics"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1149","DOI":"10.1109\/TNSRE.2019.2913400","article-title":"Spectral and temporal feature learning with two-stream neural networks for mental workload assessment","volume":"27","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1320","DOI":"10.1109\/12.106218","article-title":"Neural-network-based fuzzy logic control and decision system","volume":"40","author":"Lin","year":"1991","journal-title":"IEEE Trans. Comput."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/0165-0114(93)90181-G","article-title":"Fuzzy neural networks and neurocomputations","volume":"56","author":"Pedrycz","year":"1993","journal-title":"Fuzzy Sets Syst."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"733","DOI":"10.1109\/TFUZZ.2004.834803","article-title":"Supervisory recurrent fuzzy neural network control of wing rock for slender delta wings","volume":"12","author":"Lin","year":"2004","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Long, H., and Wang, X. (2009, January 10\u201311). Application of aircraft fuel fault diagnostic expert system based on fuzzy neural network. Proceedings of the 2009 WASE International Conference on Information Engineering, Taiyuan, China.","DOI":"10.1109\/ICIE.2009.9"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"5402809","DOI":"10.1155\/2017\/5402809","article-title":"Learning control of fixed-wing unmanned aerial vehicles using fuzzy neural networks","volume":"2017","author":"Kayacan","year":"2017","journal-title":"Int. J. Aerosp. Eng."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s40313-019-00465-y","article-title":"Autonomous landing of UAV based on artificial neural network supervised by fuzzy logic","volume":"30","author":"Souza","year":"2019","journal-title":"J. Control Autom. Electr. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.ins.2019.04.007","article-title":"Adaptive control for fractional order induced chaotic fuzzy cellular neural networks and its application to image encryption","volume":"491","author":"Mani","year":"2019","journal-title":"Inf. Sci."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1950003","DOI":"10.1142\/S0218213019500039","article-title":"Pulsar detection for wavelets soda and regularized fuzzy neural networks based on andneuron and robust activation function","volume":"28","author":"Torres","year":"2019","journal-title":"Int. J. Artif. Intell. Tools"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Chen, W., Xu, T., Liu, J., Wang, M., and Zhao, D. (2019). Picking robot visual servo control based on modified fuzzy neural network sliding mode algorithms. Electronics, 8.","DOI":"10.3390\/electronics8060605"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Kumar, S., Kumar, A., Bajaj, V., and Singh, G.K. (2019). An improved fuzzy min-max neural network for data classification. IEEE Trans. Fuzzy Syst.","DOI":"10.1109\/TFUZZ.2019.2924396"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.inffus.2018.07.005","article-title":"Confidence factor weighted Gaussian function induced parallel fuzzy rank-level fusion for inference and its application to face recognition","volume":"47","author":"Sing","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Verma, N., and Ghosh, A. (2019). Fuzzy inference network with mamdani fuzzy inference system. Computational Intelligence: Theories, Applications and Future Directions\u2014Volume I, Advances in Intelligent Systems and Computing, Springer.","DOI":"10.1007\/978-981-13-1132-1_29"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Jolliffe, I.T. (2011). Principal Component Analysis, Springer. [2nd ed.].","DOI":"10.1007\/978-3-642-04898-2_455"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Migneault, J., Robert, J.M., Desmarais, M.C., and Caron, S. (2007, January 7\u201310). Comparing voice with touch screen for controlling the instructor\u2019s operating station of a flight simulator. Proceedings of the 2007 IEEE International Conference on Systems, Man and Cybernetics, Montreal, QC, Canada.","DOI":"10.1109\/ICSMC.2007.4414002"},{"key":"ref_38","unstructured":"Stein, E.S. (1984). The Measurement of Pilot Performance: A Master-Journeyman Approach, Federal Aviation Administration."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Gawron, V.J. (2008). Human Performance, Workload, and Situational Awareness Measures Handbook, CRC Press. [2nd ed.].","DOI":"10.1201\/9781420064506"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Nagasawa, T., and Hagiwara, H. (November, January 31). Workload induces changes in hemodynamics, respiratory rate and heart rate variability. Proceedings of the 2016 IEEE 16th International Conference on Bioinformatics and Bioengineering (BIBE), Taichung, Taiwan.","DOI":"10.1109\/BIBE.2016.27"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"8146809","DOI":"10.1155\/2016\/8146809","article-title":"Respiratory changes in response to cognitive load: A systematic review","volume":"2016","author":"Grassmann","year":"2016","journal-title":"Neural Plast."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1016\/j.ergon.2006.04.002","article-title":"Using eye movement activity as a correlate of cognitive workload","volume":"36","author":"Ahlstrom","year":"2006","journal-title":"Int. J. Ind. Ergon."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1080\/24721840.2018.1514978","article-title":"Eye-tracking measures in aviation: A selective literature review","volume":"28","author":"Wickens","year":"2018","journal-title":"Int. J. Aerosp. Psychol."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1016\/j.apergo.2018.08.028","article-title":"Measuring mental workload using physiological measures: A systematic review","volume":"74","author":"Charles","year":"2019","journal-title":"Appl. Ergon."},{"key":"ref_45","unstructured":"Corcoran, J.P. (2019). Comparing the Effects of Mental Workload between Visual and Auditory Secondary Tasks during Laparoscopy. [Master\u2019s Thesis, Old Dominion University]."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/16\/3629\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,19]],"date-time":"2024-06-19T23:49:03Z","timestamp":1718840943000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/16\/3629"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,8,20]]},"references-count":45,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2019,8]]}},"alternative-id":["s19163629"],"URL":"http:\/\/dx.doi.org\/10.3390\/s19163629","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,8,20]]}}}
