Abstract
SLAM (Simultaneously Localization and Mapping) is the fundamental technology for the application of unmanned intelligent systems, such as underwater exploration with fish robots. But various visual challenges often occur in practical environments, severely threaten the system robustness. Currently, few research explicitly focus on visual challenges for SLAM and analyze them quantitatively, resulting in works with less comprehensiveness and generalization. Many are basically not intelligent enough in the changing real world and sometimes even infeasible for practical deployment due to the lack of accurate visual cognition in the ambient environment, as many animals do. Inspired by visual perception pathways in brains, we try to solve the problem from the view of visual cognition and propose a fully computational reliable evaluation method for general challenges to push the frontier of visual SLAM. It systematically decomposes various challenges into three relevant aspects and evaluates the perception quality with corresponding scores. Extensive experiments on different datasets demonstrate the feasibility and effectiveness of our method by a strong correlation with SLAM performance. Moreover, we automatically obtain detailed insights about challenges from quantitative evaluation, which is also important for targeted solutions. To our best knowledge, no similar works exist at present.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Benesty, J., Chen, J., Huang, Y., Cohen, I.: Pearson Correlation Coefficient, pp. 1–4. Springer, Berlin, Heidelberg (2009). https://doi.org/10.1007/978-3-642-00296-0_5
Brunner, C., Peynot, T.: Visual metrics for the evaluation of sensor data quality in outdoor perception. In: Proceedings of the 10th Performance Metrics for Intelligent Systems Workshop, pp. 1–8. PerMIS ’10, Association for Computing Machinery, New York, NY, USA (2010)
Brunner, C., Peynot, T.: Perception quality evaluation with visual and infrared cameras in challenging environmental conditions. In: Khatib, O., Kumar, V., Sukhatme, G. (eds.) Experimental Robotics. Springer Tracts in Advanced Robotics, vol. 79, pp. 711–725. Springer, Berlin, Heidelberg (2014). https://doi.org/10.1007/978-3-642-28572-1_49
Burri, M., Nikolic, J., Gohl, P., Schneider, T., Rehder, J., Omari, S., Achtelik, M.W., Siegwart, R.: The euroc micro aerial vehicle datasets. Int. J. Robot. Res. 35(10), 1157–1163 (2016)
Campos, C., Elvira, R., Rodriguez, J.J.G., Montiel, J.M., Tardos, J.D.: ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap slam. IEEE Trans. Robot. 37(6), 1874–1890 (2021)
Carrillo, H., Reid, I., Castellanos, J.A.: On the comparison of uncertainty criteria for active slam. In: IEEE International Conference on Robotics and Automation, pp. 2080–2087 (2012)
Cepeda-Negrete, J., Sanchez-Yanez, R.E.: Gray-world assumption on perceptual color spaces. In: Klette, R., Rivera, M., Satoh, S. (eds.) PSIVT 2013. LNCS, vol. 8333, pp. 493–504. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-53842-1_42
Engel, J., Koltun, V., Cremers, D.: Direct sparse odometry. IEEE Trans. Pattern Anal. Mach. Intell. 40(3), 611–625 (2018)
Ferrera, M., Creuze, V., Moras, J., Trouvé-Peloux, P.: AQUALOC: an underwater dataset for visual-inertial-pressure localization. Int. J. Robot. Res. 38(14), 1549–1559 (2019)
Gadkari, D.: Image quality analysis using GLCM (2004)
Geiger, A., Lenz, P., Stiller, C., Urtasun, R.: Vision meets robotics: the KITTI dataset. Int. J. Robot. Res. 32(11), 1231–1237 (2013)
Kim, P., Coltin, B., Alexandrov, O., Kim, H.J.: Robust visual localization in changing lighting conditions. In: IEEE International Conference on Robotics and Automation, pp. 5447–5452 (2017)
Ma, K., Liu, W., Zhang, K., Duanmu, Z., Wang, Z., Zuo, W.: End-to-end blind image quality assessment using deep neural networks. IEEE Trans. Image Process. 27(3), 1202–1213 (2018)
Ma, P., et al.: Multiscale superpixelwise prophet model for noise-robust feature extraction in hyperspectral images. IEEE Trans. Geosci. Remote Sens. 61, 1–12 (2023)
Moorthy, A.K., Bovik, A.C.: Blind image quality assessment: From natural scene statistics to perceptual quality. IEEE Trans. Image Process. 20(12), 3350–3364 (2011)
Mur-Artal, R., Montiel, J.M.M., Tardós, J.D.: Orb-slam: a versatile and accurate monocular slam system. IEEE Trans. Robot. 31(5), 1147–1163 (2015)
Santamaria-Navarro, A., Thakker, R., Fan, D.D., Morrell, B., Agha-mohammadi, A.A.: Towards resilient autonomous navigation of drones. In: Asfour, T., Yoshida, E., Park, J., Christensen, H., Khatib, O. (eds.) Robotics Research. ISRR 2019. LNCS. Springer Proceedings in Advanced Robotics, vol. 20, pp. 922–937. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-95459-8_57
Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D SLAM systems. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 573–580 (2012)
Teed, Z., Deng, J.: Droid-slam: deep visual slam for monocular, stereo, and rgb-d cameras. Adv. Neural Inf. Process. Syst. 34, 16558–16569 (2021)
Tranzatto, M., et al.: Cerberus: autonomous legged and aerial robotic exploration in the tunnel and urban circuits of the darpa subterranean challenge. arXiv preprint arXiv:2201.07067 (2022)
Xiang, T., Yang, Y., Guo, S.: Blind night-time image quality assessment: subjective and objective approaches. IEEE Trans. Multimed. 22(5), 1259–1272 (2020)
Xiong, J., Wang, J., Heidrich, W., Nayar, S.: Seeing in extra darkness using a deep-red flash. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9995–10004 (2021)
Yang, N., Zhong, Q., Li, K., Cong, R., Zhao, Y., Kwong, S.: A reference-free underwater image quality assessment metric in frequency domain. Signal Process. Image Commun. 94, 116218 (2021)
Zhang, J., Singh, S.: Enabling aggressive motion estimation at low-drift and accurate mapping in real-time. In: IEEE International Conference on Robotics and Automation, pp. 5051–5058 (2017)
Zhang, Z., Scaramuzza, D.: A tutorial on quantitative trajectory evaluation for visual(-inertial) odometry. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 7244–7251 (2018)
Zhao, X.: The genshin impact dataset (GID) for slam (2023). https://github.com/zhaoxuhui/Genshin-Impact-Dataset
Acknowledgements
This research is partially supported by the National Natural Science Foundation of China Major Program (Grant No. 42192580, 42192583), Hubei Province Natural Science Foundation (Grant No. 2021CFA088 and 2020-CFA003), the Science and Technology Major Project (Grant No. 2021AAA010), and Wuhan University - Huawei Geoinformatics Innovation Laboratory. Numerical calculations are supported by Supercomputing Center of Wuhan University.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Zhao, X., Gao, Z., Li, H., Li, C., Chen, J., Yi, H. (2024). How Challenging is a Challenge for SLAM? An Answer from Quantitative Visual Evaluation. In: Ren, J., et al. Advances in Brain Inspired Cognitive Systems. BICS 2023. Lecture Notes in Computer Science(), vol 14374. Springer, Singapore. https://doi.org/10.1007/978-981-97-1417-9_17
Download citation
DOI: https://doi.org/10.1007/978-981-97-1417-9_17
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-1416-2
Online ISBN: 978-981-97-1417-9
eBook Packages: Computer ScienceComputer Science (R0)