Abstract
In medical imaging analysis, deep learning has shown promising results. We frequently rely on volumetric data to segment medical images, necessitating the use of 3D architectures, which are commended for their capacity to capture interslice context. However, because of the 3D convolutions, max pooling, up-convolutions, and other operations utilized in these networks, these architectures are often more inefficient in terms of time and computation than their 2D equivalents. Furthermore, there are few 3D pretrained model weights, and pretraining is often tricky. We present a simple yet effective 2D method to handle 3D data while efficiently embedding the 3D knowledge during training. We propose transforming volumetric data into 2D super images and segmenting with 2D networks to solve these challenges. Our method generates a super-resolution image by stitching slices side by side in the 3D image. We expect deep neural networks to capture and learn these properties spatially despite losing depth information. This work aims to present a novel perspective when dealing with volumetric data, and we test the hypothesis using CNN and ViT networks as well as self-supervised pretraining. While attaining equal, if not superior, results to 3D networks utilizing only 2D counterparts, the model complexity is reduced by around threefold. Because volumetric data is relatively scarce, we anticipate that our approach will entice more studies, particularly in medical imaging analysis.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ahn, B.B.: The compact 3D convolutional neural network for medical images. Standford University (2017)
Andrearczyk, V., Oreiller, V., Depeursinge, A.: Head and neck tumor segmentation in pet/ct. Medical Image Analysis (2021), www.aicrowd.com/challenges/miccai-2021-hecktor
Baumgartner, C.F., Koch, L.M., Pollefeys, M., Konukoglu, E.: An exploration of 2D and 3D deep learning techniques for cardiac MR image segmentation. In: Pop, M., Sermesant, M., Jodoin, P.-M., Lalande, A., Zhuang, X., Yang, G., Young, A., Bernard, O. (eds.) STACOM 2017. LNCS, vol. 10663, pp. 111–119. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75541-0_12
Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3D U-Net: learning dense volumetric segmentation from sparse annotation. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 424–432. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_49
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Fan, Q., Chen, C.F., Panda, R.: Can an image classifier suffice for action recognition? In: International Conference on Learning Representations (2021)
Feng, X., Tustison, N.J., Patel, S.H., Meyer, C.H.: Brain tumor segmentation using an ensemble of 3D u-nets and overall survival prediction using radiomic features. Front. Comput. Neurosci. 14, 25 (2020), https://doi.org/10.3389/fncom.2020.00025, https://www.frontiersin.org/article/10.3389/fncom.2020.00025
Hatamizadeh, A., et al.: Swin unetr: swin transformers for semantic segmentation of brain tumors in MRI images. In: International MICCAI Brainlesion Workshop. pp. 272–284. Springer (2022)
Iantsen, A., Visvikis, D., Hatt, M.: Squeeze-and-excitation normalization for automated delineation of head and neck primary tumors in combined PET and CT Images. In: Andrearczyk, V., Oreiller, V., Depeursinge, A. (eds.) HECKTOR 2020. LNCS, vol. 12603, pp. 37–43. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-67194-5_4
Milletari, F., Navab, N., Ahmadi, S.A.: V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 565–571. IEEE (2016)
Patravali, J., Jain, S., Chilamkurthy, S.: 2D-3D fully convolutional neural networks for cardiac MR segmentation. In: Pop, M., Sermesant, M., Jodoin, P.-M., Lalande, A., Zhuang, X., Yang, G., Young, A., Bernard, O. (eds.) STACOM 2017. LNCS, vol. 10663, pp. 130–139. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-75541-0_14
Roth, H.R., et al.: A New 2.5D representation for lymph node detection using random sets of deep convolutional neural network observations. In: Golland, P., Hata, N., Barillot, C., Hornegger, J., Howe, R. (eds.) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2014: 17th International Conference, Boston, MA, USA, September 14-18, 2014, Proceedings, Part I, pp. 520–527. Springer International Publishing, Cham (2014). https://doi.org/10.1007/978-3-319-10404-1_65
Xiong, Z., et al.: A global benchmark of algorithms for segmenting the left atrium from late gadolinium-enhanced cardiac magnetic resonance imaging. Med. Image Anal. 67, 101832 (2021)
Yakobchuk, V.: Smart male practitioner examining mri scan image stock photo, stock.adobe.com/images/smart-male-practitioner-examining-mri-scan-image/169384005
Yang, J., et al.: Reinventing 2D convolutions for 3D images. IEEE J. Biomed. Health Inform. 25(8), 3009–3018 (2021)
Zhang, Y., Liu, H., Hu, Q.: Transfuse: Fusing transformers and CNNs for medical image segmentation. CoRR abs/2102.08005 (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sobirov, I., Saeed, N., Yaqub, M. (2024). Super Images - A New 2D Perspective on 3D Medical Imaging Analysis. In: Waiter, G., Lambrou, T., Leontidis, G., Oren, N., Morris, T., Gordon, S. (eds) Medical Image Understanding and Analysis. MIUA 2023. Lecture Notes in Computer Science, vol 14122. Springer, Cham. https://doi.org/10.1007/978-3-031-48593-0_24
Download citation
DOI: https://doi.org/10.1007/978-3-031-48593-0_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-48592-3
Online ISBN: 978-3-031-48593-0
eBook Packages: Computer ScienceComputer Science (R0)