{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,9,2]],"date-time":"2024-09-02T17:49:43Z","timestamp":1725299383847},"reference-count":30,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2022,2,16]],"date-time":"2022-02-16T00:00:00Z","timestamp":1644969600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"Machine Learning (ML) methods have become state of the art in radar signal processing, particularly for classification tasks (e.g., of different human activities). Radar classification can be tedious to implement, though, due to the limited size and diversity of the source dataset, i.e., the data measured once for initial training of the Machine Learning algorithms. In this work, we introduce the algorithm Radar Activity Classification with Perceptual Image Transformation (RACPIT), which increases the accuracy of human activity classification while lowering the dependency on limited source data. In doing so, we focus on the augmentation of the dataset by synthetic data. We use a human radar reflection model based on the captured motion of the test subjects performing activities in the source dataset, which we recorded with a video camera. As the synthetic data generated by this model still deviates too much from the original radar data, we implement an image transformation network to bring real data close to their synthetic counterpart. We leverage these artificially generated data to train a Convolutional Neural Network for activity classification. We found that by using our approach, the classification accuracy could be increased by up to 20%, without the need of collecting more real data.<\/jats:p>","DOI":"10.3390\/s22041519","type":"journal-article","created":{"date-parts":[[2022,2,17]],"date-time":"2022-02-17T02:36:24Z","timestamp":1645065384000},"page":"1519","source":"Crossref","is-referenced-by-count":6,"title":["Improving Radar Human Activity Classification Using Synthetic Data with Image Transformation"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-1284-4951","authenticated-orcid":false,"given":"Rodrigo","family":"Hernang\u00f3mez","sequence":"first","affiliation":[{"name":"Fraunhofer Heinrich Hertz Institute, 10587 Berlin, Germany"}]},{"given":"Tristan","family":"Visentin","sequence":"additional","affiliation":[{"name":"Fraunhofer Heinrich Hertz Institute, 10587 Berlin, Germany"}]},{"ORCID":"http:\/\/orcid.org\/0000-0003-4322-834X","authenticated-orcid":false,"given":"Lorenzo","family":"Servadei","sequence":"additional","affiliation":[{"name":"Infineon Technologies AG, 85579 Munich, Germany"},{"name":"Department of Electrical and Computer Engineering, Technical University of Munich, 80333 Munich, Germany"}]},{"given":"Hamid","family":"Khodabakhshandeh","sequence":"additional","affiliation":[{"name":"Fraunhofer Heinrich Hertz Institute, 10587 Berlin, Germany"}]},{"ORCID":"http:\/\/orcid.org\/0000-0003-3829-4668","authenticated-orcid":false,"given":"S\u0142awomir","family":"Sta\u0144czak","sequence":"additional","affiliation":[{"name":"Fraunhofer Heinrich Hertz Institute, 10587 Berlin, Germany"},{"name":"Faculty IV, Electrical Engineering and Computer Science, Technical University of Berlin, 10587 Berlin, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2022,2,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"7001804","DOI":"10.1109\/LSENS.2018.2882642","article-title":"Robust Gesture Recognition Using Millimetric-Wave Radar System","volume":"2","author":"Hazra","year":"2018","journal-title":"IEEE Sens. Lett."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Stephan, M., and Santra, A. (2019, January 16\u201319). Radar-Based Human Target Detection using Deep Residual U-Net for Smart Home Applications. Proceedings of the 2019 18th IEEE International Conference on Machine Learning and Applications (ICMLA), Boca Raton, FL, USA.","DOI":"10.1109\/ICMLA.2019.00035"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Stephan, M., Stadelmayer, T., Santra, A., Fischer, G., Weigel, R., and Lurz, F. (2021, January 10\u201315). Radar Image Reconstruction from Raw ADC Data using Parametric Variational Autoencoder with Domain Adaptation. Proceedings of the 2020 25th International Conference on Pattern Recognition (ICPR), Milan, Italy.","DOI":"10.1109\/ICPR48806.2021.9412858"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Redko, I., Morvant, E., Habrard, A., Sebban, M., and Bennani, Y. (2019). Advances in Domain Adaptation Theory, Elsevier.","DOI":"10.1016\/B978-1-78548-236-6.50002-7"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1109\/LGRS.2015.2491329","article-title":"Human Detection and Activity Classification Based on Micro-Doppler Signatures Using Deep Convolutional Neural Networks","volume":"13","author":"Kim","year":"2016","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Shah, S.A., and Fioranelli, F. (2019, January 23\u201327). Human Activity Recognition: Preliminary Results for Dataset Portability using FMCW Radar. Proceedings of the 2019 International Radar Conference (RADAR), Toulon, France.","DOI":"10.1109\/RADAR41533.2019.171307"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1109\/LGRS.2019.2917301","article-title":"Unsupervised Adversarial Domain Adaptation for Micro-Doppler Based Human Activity Classification","volume":"17","author":"Du","year":"2020","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Li, X., Jing, X., and He, Y. (2020, January 21\u201325). Unsupervised Domain Adaptation for Human Activity Recognition in Radar. Proceedings of the 2020 IEEE Radar Conference (RadarConf20), Florence, Italy.","DOI":"10.1109\/RadarConf2043947.2020.9266433"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Khodabakhshandeh, H., Visentin, T., Hernang\u00f3mez, R., and P\u00fctz, M. (2021, January 21\u201322). Domain Adaptation Across Configurations of FMCW Radar for Deep Learning Based Human Activity Classification. Proceedings of the 2021 21st International Radar Symposium (IRS), Berlin, Germany.","DOI":"10.23919\/IRS51887.2021.9466179"},{"key":"ref_10","unstructured":"Chen, V.C. (2011). The Micro-Doppler Effect in Radar, Artech House."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Johnson, J., Alahi, A., and Fei-Fei, L. (2016). Perceptual Losses for Real-Time Style Transfer and Super-Resolution. arXiv.","DOI":"10.1007\/978-3-319-46475-6_43"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Hernang\u00f3mez, R., Santra, A., and Sta\u0144czak, S. (2019, January 23\u201327). Human Activity Classification with Frequency Modulated Continuous Wave Radar Using Deep Convolutional Neural Networks. Proceedings of the 2019 International Radar Conference (RADAR), Toulon, France.","DOI":"10.1109\/RADAR41533.2019.171243"},{"key":"ref_13","first-page":"932","article-title":"A Study on Feature Processing Schemes for Deep-Learning-Based Human Activity Classification Using Frequency-Modulated Continuous-Wave Radar","volume":"15","author":"Santra","year":"2020","journal-title":"IET Radar Sonar Navig."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Gurbuz, S.Z., Rahman, M.M., Kurtoglu, E., Macks, T., and Fioranelli, F. (2020). Cross-frequency training with adversarial learning for radar micro-Doppler signature classification (Rising Researcher). Radar Sensor Technology XXIV, International Society for Optics and Photonics.","DOI":"10.1117\/12.2559155"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Gusland, D., Christiansen, J.M., Torvik, B., Fioranelli, F., Gurbuz, S.Z., and Ritchie, M. (2021, January 7\u201314). Open Radar Initiative: Large Scale Dataset for Benchmarking of micro-Doppler Recognition Algorithms. Proceedings of the 2021 IEEE Radar Conference (RadarConf21), Atlanta, GA, USA.","DOI":"10.1109\/RadarConf2147009.2021.9455239"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3287078","article-title":"Exploring Tangible Interactions with Radar Sensing","volume":"2","author":"Yeo","year":"2018","journal-title":"Proc. ACM Interact. Mob. Wearable Ubiquitous Technol."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zhang, A., Nowruzi, F.E., and Laganiere, R. (2021, January 26\u201328). RADDet: Range-Azimuth-Doppler based Radar Object Detection for Dynamic Road Users. Proceedings of the 2021 18th Conference on Robots and Vision (CRV), Burnaby, BC, Canada.","DOI":"10.1109\/CRV52889.2021.00021"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Stolz, M., Schubert, E., Meinl, F., Kunert, M., and Menzel, W. (2017, January 11\u201313). Multi-target reflection point model of cyclists for automotive radar. Proceedings of the 2017 European Radar Conference (EURAD), Nuremberg, Germany.","DOI":"10.23919\/EURAD.2017.8249155"},{"key":"ref_19","unstructured":"De la Torre, F., Hodgins, J., Montano, J., Valcarcel, S., Forcada, R., and Macey, J. (2009). Guide to the Carnegie Mellon University Multimodal Activity (CMU-MMAC) Database, Robotics Institute, Carnegie Mellon University."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Isola, P., Zhu, J.Y., Zhou, T., and Efros, A.A. (2017, January 21\u201326). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.632"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhu, J.Y., Park, T., Isola, P., and Efros, A.A. (2017, January 22\u201329). Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of the IEEE international conference on computer vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Richards, M.A., Scheer, J., Holm, W.A., and Melvin, W.L. (2010). Principles of Modern Radar, Citeseer.","DOI":"10.1049\/SBRA021E"},{"key":"ref_23","unstructured":"Manolakis, D.G., Ingle, V.K., and Kogon, S.M. (2005). Statistical and Adaptive Signal Processing: Spectral Estimation, Signal Modeling, Adaptive Filtering and Array Processing, Artech House Signal Processing Library, Artech House."},{"key":"ref_24","unstructured":"Wu, Y., Kirillov, A., Massa, F., Lo, W.Y., and Girshick, R. (2022, February 14). Detectron2. Available online: https:\/\/github.com\/facebookresearch\/detectron2."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"344","DOI":"10.1007\/BF01901021","article-title":"A global human walking model with real-time kinematic personification","volume":"6","author":"Boulic","year":"1990","journal-title":"Vis. Comput."},{"key":"ref_26","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, The MIT Press. Adaptive Computation and Machine Learning."},{"key":"ref_27","unstructured":"Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_28","unstructured":"Wu, X.Z., and Zhou, Z.H. (2017, January 6\u201311). A unified view of multi-label performance measures. Proceedings of the 34th International Conference on Machine Learning (ICML\u201917), Sydney, NSW, Australia."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Brodersen, K.H., Ong, C.S., Stephan, K.E., and Buhmann, J.M. (2010, January 23\u201326). The Balanced Accuracy and Its Posterior Distribution. Proceedings of the 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey.","DOI":"10.1109\/ICPR.2010.764"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Guler, R.A., Neverova, N., and Kokkinos, I. (2018, January 18\u201323). DensePose: Dense Human Pose Estimation in the Wild. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00762"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/4\/1519\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,26]],"date-time":"2024-07-26T04:32:40Z","timestamp":1721968360000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/4\/1519"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,16]]},"references-count":30,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["s22041519"],"URL":"https:\/\/doi.org\/10.3390\/s22041519","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,16]]}}}
