Abstract
Data models are required to provide the semantics of the underlying data stream for in situ visualization. In this paper we describe a set of metrics for such a data model that are useful in meeting the needs of the scientific community for visualization. We then present Fides, a library that provides a schema for the VTK-m data model, and uses the ADIOS middleware library for access to streaming data. We present four use cases of Fides in different scientific workflows, and provide an evaluation of each use case against our metrics.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Cloverleaf3D. http://uk-mac.github.io/CloverLeaf3D/. Accessed 19 Dec 2018
VTK-m users guide. https://m.vtk.org/images/c/c8/VTKmUsersGuide.pdf. Accessed 17 June 2021
Ayachit, U., et al.: ParaView catalyst: enabling in situ data analysis and visualization. In: Proceedings of the First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization, pp. 25–29. ACM (2015)
Ayachit, U., et al.: Performance analysis, design considerations, and applications of extreme-scale in situ infrastructures. In: ACM/IEEE International Conference for High Performance Computing, Networking, Storage and Analysis (SC16), Salt Lake City, UT, USA, November 2016. https://doi.org/10.1109/SC.2016.78. lBNL-1007264
Childs, H., Bennett, J., Garth, C., Hentschel, B.: In situ visualization for computational science. IEEE Comput. Graph. Appl. 39(6), 76–85 (2019)
Childs, H., et al.: VisIt: an end-user tool for visualizing and analyzing very large data. In: Bethel, E.W., Childs, H., Hansen, C. (eds.) High Performance Visualization-Enabling Extreme-Scale Scientific Insight, pp. 357–372. Chapman & Hall, CRC Computational Science, CRC Press/Francis-Taylor Group, Boca Raton, November 2012. http://www.crcpress.com/product/isbn/9781439875728. LBNL-6320E
Clarke, l.J., Mark, E.: Enhancements to the extensible data model and format (XDMF). In: 2007 DoD High Performance Computing Modernization Program Users Group Conference, pp. 322–327 (2007). https://doi.org/10.1109/HPCMP-UGC.2007.30
De Melo, A.C.: The new Linux ‘perf’ tools. In: Slides from Linux Kongress, vol. 18, pp. 1–42 (2010)
Godoy, W., et al.: Adios 2: The adaptable input output system. a framework for high-performance data management. SoftwareX 12, 100561 (2020). https://doi.org/10.1016/j.softx.2020.100561
Huebl, A., et al.: openPMD: a meta data standard for particle and mesh based data (2015). https://doi.org/10.5281/zenodo.591699
Klasky, S., et al.: A view from ORNL: scientific data research opportunities in the big data age. In: 38th IEEE International Conference on Distributed Computing Systems, ICDCS 2018, Vienna, Austria, 2–6 July 2018, pp. 1357–1368. IEEE Computer Society (2018). https://doi.org/10.1109/ICDCS.2018.00136
Ku, S., et al.: A fast low-to-high confinement mode bifurcation dynamics in the boundary-plasma gyrokinetic code XGC1. Phys. Plasmas 25(5), 056107 (2018). https://doi.org/10.1063/1.5020792
Larsen, M., et al.: The ALPINE in situ infrastructure: ascending from the ashes of strawman. In: Proceedings of the In Situ Infrastructures on Enabling Extreme-Scale Analysis and Visualization, pp. 42–46. ACM (2017)
Lawrence Livermore National Laboratory: Blueprint. https://llnl-conduit.readthedocs.io/en/latest/index.html. Accessed 18 June 2020
Lawrence Livermore National Laboratory: Conduit. https://llnl-conduit.readthedocs.io/en/latest/blueprint.html. Accessed 9 June 2020
Mallinson, A., et al.: CloverLeaf: preparing hydrodynamics codes for exascale. The Cray User Group 2013 (2013)
Meredith, J.S., Ahern, S., Pugmire, D., Sisneros, R.: EAVL: the extreme-scale analysis and visualization library. In: Childs, H., Kuhlen, T., Marton, F. (eds.) Eurographics Symposium on Parallel Graphics and Visualization. The Eurographics Association (2012). https://doi.org/10.2312/EGPGV/EGPGV12/021-030
Moreland, K., Ayachit, U., Geveci, B., Ma, K.: Dax toolkit: a proposed framework for data analysis and visualization at extreme scale. In: Rogers, D.H., Silva, C.T. (eds.) IEEE Symposium on Large Data Analysis and Visualization, LDAV 2011, Providence, Rhode Island, USA, 23–24 October 2011, pp. 97–104. IEEE Computer Society (2011). https://doi.org/10.1109/LDAV.2011.6092323
Moreland, K., et al.: VTK-m: accelerating the visualization toolkit for massively threaded architectures. IEEE Comput. Graph. Appl. 36(3), 48–58 (2016). https://doi.org/10.1109/MCG.2016.48
ORNL: Grey-Scott simulation code (2018). https://github.com/suchyta1/adiosvm/blob/cpp/Tutorial/gray-scott/simulation/gray-scott.cpp. Accessed 9 Apr 2021
Pugmire, D., et al.: Visualization as a service for scientific data. In: Nichols, J., Verastegui, B., Maccabe, A.B., Hernandez, O., Parete-Koon, S., Ahearn, T. (eds.) SMC 2020. CCIS, vol. 1315, pp. 157–174. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-63393-6_11
Schroeder, W.J., Lorensen, B., Martin, K.: The visualization toolkit: an object-oriented approach to 3D graphics. Kitware (2004)
Tchoua, R., et al.: Adios visualization schema: a first step towards improving interdisciplinary collaboration in high performance computing. In: 2013 IEEE 9th International Conference on e-Science, pp. 27–34 (2013). https://doi.org/10.1109/eScience.2013.24
Vazhkudai, S.S., et al.: The design, deployment, and evaluation of the coral pre-exascale systems. In: SC18: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 661–672 (2018)
Wang, B., et al.: Kinetic turbulence simulations at extreme scale on leadership-class systems. In: Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, pp. 1–12 (2013)
Whitlock, B., Favre, J.M., Meredith, J.S.: Parallel in situ coupling of simulation with a fully featured visualization system. In: Kuhlen, T., et al. (eds.) Eurographics Symposium on Parallel Graphics and Visualization. The Eurographics Association (2011). https://doi.org/10.2312/EGPGV/EGPGV11/101-109
Acknowledgements
This work was supported by the U.S. Department of Energy, Office of Science, Office of Fusion Energy Sciences under Award Number DE-SC0018054 and the Scientific Discovery through Advanced Computing (SciDAC) program in U.S. Department of Energy. This work also used resources of the Oak Ridge Leadership Computing Facility, which is a U.S. Department of Energy, Office of Science User Facility.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Pugmire, D. et al. (2021). Fides: A General Purpose Data Model Library for Streaming Data. In: Jagode, H., Anzt, H., Ltaief, H., Luszczek, P. (eds) High Performance Computing. ISC High Performance 2021. Lecture Notes in Computer Science(), vol 12761. Springer, Cham. https://doi.org/10.1007/978-3-030-90539-2_34
Download citation
DOI: https://doi.org/10.1007/978-3-030-90539-2_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-90538-5
Online ISBN: 978-3-030-90539-2
eBook Packages: Computer ScienceComputer Science (R0)