{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,10,4]],"date-time":"2024-10-04T04:22:41Z","timestamp":1728015761942},"reference-count":66,"publisher":"SAGE Publications","issue":"6","license":[{"start":{"date-parts":[[2020,7,9]],"date-time":"2020-07-09T00:00:00Z","timestamp":1594252800000},"content-version":"vor","delay-in-days":366,"URL":"http:\/\/www.sagepub.com\/licence-information-for-chorus"}],"funder":[{"DOI":"10.13039\/100000001","name":"DOE\/NNSA Exascale Computing Project","doi-asserted-by":"publisher","award":["17-SC-20-SC"],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of High Performance Computing Applications"],"published-print":{"date-parts":[[2019,11]]},"abstract":" Architectural and technological trends of systems used for scientific computing call for a significant reduction of scientific data sets that are composed mainly of floating-point data. This article surveys and presents experimental results of currently identified use cases of generic lossy compression to address the different limitations of scientific computing systems. The article shows from a collection of experiments run on parallel systems of a leadership facility that lossy data compression not only can reduce the footprint of scientific data sets on storage but also can reduce I\/O and checkpoint\/restart times, accelerate computation, and even allow significantly larger problems to be run than without lossy compression. These results suggest that lossy compression will become an important technology in many aspects of high performance scientific computing. Because the constraints for each use case are different and often conflicting, this collection of results also indicates the need for more specialization of the compression pipelines. <\/jats:p>","DOI":"10.1177\/1094342019853336","type":"journal-article","created":{"date-parts":[[2019,7,10]],"date-time":"2019-07-10T03:18:23Z","timestamp":1562728703000},"page":"1201-1220","update-policy":"http:\/\/dx.doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":88,"title":["Use cases of lossy compression for floating-point data in scientific data sets"],"prefix":"10.1177","volume":"33","author":[{"ORCID":"http:\/\/orcid.org\/0000-0002-7890-3934","authenticated-orcid":false,"given":"Franck","family":"Cappello","sequence":"first","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"},{"name":"Department of Computer Science, University of Illinois Urbana-Champaign, Urbana, IL, USA"}]},{"given":"Sheng","family":"Di","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"}]},{"given":"Sihuan","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of California, Riverside, CA, USA"}]},{"given":"Xin","family":"Liang","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, University of California, Riverside, CA, USA"}]},{"given":"Ali Murat","family":"Gok","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, Northwestern University, Evanston, IL, USA"}]},{"given":"Dingwen","family":"Tao","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Alabama, Tuscaloosa, AL, USA"}]},{"given":"Chun Hong","family":"Yoon","sequence":"additional","affiliation":[{"name":"Linac Coherent Light Source, SLAC National Accelerator Laboratory, Menlo Park, CA, USA"}]},{"given":"Xin-Chuan","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Chicago, Chicago, IL, USA"}]},{"given":"Yuri","family":"Alexeev","sequence":"additional","affiliation":[{"name":"Argonne National Laboratory, Lemont, IL, USA"}]},{"given":"Frederic T","family":"Chong","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Chicago, Chicago, IL, USA"}]}],"member":"179","published-online":{"date-parts":[[2019,7,9]]},"reference":[{"key":"bibr1-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1007\/s00791-018-00303-9"},{"key":"bibr2-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-67630-2_3"},{"key":"bibr3-1094342019853336","doi-asserted-by":"publisher","DOI":"10.2172\/1483828"},{"key":"bibr4-1094342019853336","unstructured":"Ballester-Ripoll R, Lindstrom P, Pajarola R (2018) TTHRESH: Tensor Compression for Multidimensional Visual Data. ArXiv e-prints. Available at: https:\/\/arxiv.org\/abs\/1806.05952 (accessed 15 June 2018)."},{"key":"bibr5-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1145\/2063384.2063427"},{"key":"bibr6-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1190\/geo2015-0653.1"},{"key":"bibr7-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1177\/1094342018762036"},{"key":"bibr8-1094342019853336","doi-asserted-by":"crossref","unstructured":"Clyne J, Mininni P, Norton A, et al. (2007) Interactive desktop analysis of high resolution simulations: application to turbulent plume dynamics and current sheet formation. New Journal of Physics 9(8): 301. Available at: http:\/\/stacks.iop.org\/1367-2630\/9\/i=8\/a=301.","DOI":"10.1088\/1367-2630\/9\/8\/301"},{"key":"bibr9-1094342019853336","doi-asserted-by":"publisher","DOI":"10.17487\/rfc1952"},{"key":"bibr10-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2016.11"},{"key":"bibr11-1094342019853336","doi-asserted-by":"crossref","unstructured":"Domingos Barbosa, Jo\u00e3o Paulo Barraca, Dalmiro Maia, et al. (2016) Power monitoring and control for large scale projects: SKA, a case study. In: Proceedings of the SPIE 9910, Observatory Operations: Strategies, Processes, and Systems VI, 99100J, Edinburgh, United Kingdom, 15 July 2016. https:\/\/doi.org\/10.1117\/12.2234496.","DOI":"10.1117\/12.2234496"},{"key":"bibr12-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1109\/DCC.2000.838221"},{"key":"bibr13-1094342019853336","doi-asserted-by":"crossref","unstructured":"Fornek TE (2017) Advanced photon source upgrade project preliminary design report. DOI: 10.2172\/1423830.","DOI":"10.2172\/1423830"},{"key":"bibr14-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-64203-1_1"},{"key":"bibr15-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1145\/3126908.3126910"},{"key":"bibr16-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2015.116"},{"key":"bibr17-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2012.2189787"},{"key":"bibr18-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1109\/CLUSTER.2018.00013"},{"volume-title":"Lossy Compression of Scientific Data Via Wavelets and Vector Quantization","year":"1997","author":"Goldschneider JR","key":"bibr19-1094342019853336"},{"key":"bibr20-1094342019853336","unstructured":"Habib S (2017) Cosmological simulations with HACC. Available at: https:\/\/cpb-us-e1.wpmucdn.com\/sites.northwestern.edu\/dist\/8\/307\/files\/2017\/08\/habib_hacc_cofi_2017-1wub1ox.pdf (accessed 7 February 2017)."},{"key":"bibr21-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1145\/3015569"},{"key":"bibr22-1094342019853336","unstructured":"HDF5 Filter Plugin (2018) Available at: https:\/\/support.hdfgroup.org\/services\/filters.html."},{"key":"bibr23-1094342019853336","unstructured":"HDF5 Library (2018) Available at: https:\/\/www.hdfgroup.org\/solutions\/hdf5\/."},{"key":"bibr24-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1175\/BAMS-D-12-00121.1"},{"key":"bibr25-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1155\/2013\/341672"},{"key":"bibr26-1094342019853336","doi-asserted-by":"publisher","DOI":"10.5194\/gmd-9-4381-2016"},{"key":"bibr27-1094342019853336","unstructured":"Kukreja N, H\u00fcckelheim J, Louboutin M, et al. (2018) Combining checkpointing and data compression for large scale seismic inversion. CoRR abs\/1810.05268. Available at: http:\/\/arxiv.org\/abs\/1810.05268 (accessed 11 October 2018)."},{"key":"bibr28-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-23400-2_34"},{"key":"bibr29-1094342019853336","unstructured":"LCRC Bebop cluster (2018) Available at: https:\/\/www.lcrc.anl.gov\/systems\/resources\/bebop. Online."},{"key":"bibr30-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2018.8622101"},{"key":"bibr31-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13336"},{"key":"bibr32-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2018.8622520"},{"key":"bibr33-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2014.2346458"},{"key":"bibr34-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2006.143"},{"volume-title":"Proceedings, 37th international free electron laser conference (FEL 2015)","year":"2015","author":"Marcus G","key":"bibr35-1094342019853336"},{"key":"bibr36-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1038\/ncomms13388"},{"key":"bibr37-1094342019853336","unstructured":"McCalpin J (2016) Memory bandwidth and system balance in hpc systems. In: Invited talk at SC16. Available at: http:\/\/sites.utexas.edu\/jdm4372\/tag\/stream-benchmark\/ (accessed 7 October 2017)."},{"key":"bibr38-1094342019853336","doi-asserted-by":"publisher","DOI":"10.2172\/984082"},{"key":"bibr39-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1016\/S0168-9002(03)00343-7"},{"volume-title":"Quantum Computation and Quantum Information","year":"2002","author":"Nielsen MA","key":"bibr40-1094342019853336"},{"key":"bibr41-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1051\/0004-6361\/201629565"},{"key":"bibr42-1094342019853336","unstructured":"Patauner C (2011) Lossy and lossless data compression of data from high energy physics experiments. PhD Dissertation: CERN-THESIS-2011-211."},{"key":"bibr43-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1016\/j.ascom.2014.06.003"},{"key":"bibr44-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1016\/j.nima.2006.06.056"},{"key":"bibr45-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2015.67"},{"key":"bibr46-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1002\/jcc.540141112"},{"issue":"2","key":"bibr47-1094342019853336","first-page":"76","volume":"1","author":"Son S","year":"2014","journal-title":"Supercomputing Frontiers and Innovations"},{"key":"bibr48-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-67630-2_4"},{"key":"bibr49-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1109\/BigData.2017.8257962"},{"key":"bibr50-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2017.115"},{"key":"bibr51-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1145\/3208040.3208050"},{"volume-title":"On implementing MPI-IO portably and with high performance","year":"1998","author":"Thakur R","key":"bibr52-1094342019853336"},{"key":"bibr53-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1186\/s40679-016-0037-7"},{"key":"bibr54-1094342019853336","unstructured":"Turner A (2017) Parallel I\/O performance. Available at: https:\/\/www.archer.ac.uk\/training\/virtual\/2017-02-08-Parallel-IO\/2017_02_ParallelIO_ARCHERWebinar.pdf (accessed 8 February 2017)."},{"key":"bibr55-1094342019853336","unstructured":"VAPOR (2018) Available at: https:\/\/www.vapor.ucar.edu\/. Online."},{"key":"bibr56-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1016\/j.ascom.2015.05.003"},{"key":"bibr57-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1017\/pasa.2017.34"},{"key":"bibr58-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1109\/30.125072"},{"key":"bibr59-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2009.2013518"},{"volume-title":"Proceedings of the 4th USENIX conference on file and storage technologies (FAST)","year":"2005","author":"Welch B","key":"bibr60-1094342019853336"},{"key":"bibr61-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1109\/CLUSTER.2011.80"},{"key":"bibr62-1094342019853336","unstructured":"Wu XC, Di S, Cappello F, et al. (2018a) Amplitude-aware lossy compression for quantum circuit simulation. In: Proceedings of the 4th international workshop on data reduction for big scientific data (DRBSD-4). Arxiv: https:\/\/arxiv.org\/abs\/1811.05140."},{"key":"bibr63-1094342019853336","unstructured":"Wu XC, Di S, Cappello F, et al. (2018b) Memory-efficient quantum circuit simulation by using lossy data compression. In: The 3rd international workshop on Post-Moore Era Supercomputing (PMES) in conjunction with IEEE\/ACM 29th international conference for High Performance Computing, Networking, Storage and Analysis (SC2018). Arxiv: https:\/\/arxiv.org\/abs\/1811.05630."},{"key":"bibr64-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1038\/sdata.2017.55"},{"key":"bibr65-1094342019853336","doi-asserted-by":"publisher","DOI":"10.1145\/361147.361115"},{"key":"bibr66-1094342019853336","unstructured":"Zstandard compressor (2018) Available at: https:\/\/facebook.github.io\/zstd\/."}],"container-title":["The International Journal of High Performance Computing Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342019853336","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/1094342019853336","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342019853336","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342019853336","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,3]],"date-time":"2024-10-03T16:58:34Z","timestamp":1727974714000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/1094342019853336"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,9]]},"references-count":66,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2019,11]]}},"alternative-id":["10.1177\/1094342019853336"],"URL":"http:\/\/dx.doi.org\/10.1177\/1094342019853336","relation":{},"ISSN":["1094-3420","1741-2846"],"issn-type":[{"type":"print","value":"1094-3420"},{"type":"electronic","value":"1741-2846"}],"subject":[],"published":{"date-parts":[[2019,7,9]]}}}
