iBet uBet web content aggregator. Adding the entire web to your favor.
iBet uBet web content aggregator. Adding the entire web to your favor.



Link to original content: https://unpaywall.org/10.1016/J.FUTURE.2019.06.016
Measuring the impact of burst buffers on data-intensive scientific workflows (Journal Article) | OSTI.GOV
skip to main content
OSTI.GOV title logo U.S. Department of Energy
Office of Scientific and Technical Information

Title: Measuring the impact of burst buffers on data-intensive scientific workflows

Journal Article · · Future Generations Computer Systems
ORCiD logo [1];  [1];  [1]; ORCiD logo [1]; ORCiD logo [1]
  1. Univ. of Southern California, Los Angeles, CA (United States)
+ Show Author Affiliations

Science applications frequently produce and consume large volumes of data, but delivering this data to and from compute resources can be concerning, as parallel file system performance is not keeping up with compute and memory performance. To alter this I/O bottleneck, some systems have deployed burst buffers, but their impact on performance for real-world scientific workflow applications is still not clear. In this paper, we examine the impact of burst buffers through the remote-shared, allocatable burst buffers on the Cori system at NERSC. By running two data-intensive workflows, a high-throughput genome analysis workflow, and a subset of the SCEC high-performance CyberShake workflow, a production seismic hazard analysis workflow, we find that using burst buffers offers read and write improvements of an order of magnitude, and these improvements lead to increased job performance, and thereby increased overall workflow performance, even for long-running CPU-bound jobs.

Research Organization:
Univ. of Southern California, Los Angeles, CA (United States); Oak Ridge National Laboratory (ORNL), Oak Ridge, TN (United States); Lawrence Berkeley National Laboratory (LBNL), Berkeley, CA (United States); Rensselaer Polytechnic Inst., Troy, NY (United States)
Sponsoring Organization:
USDOE Office of Science (SC), Advanced Scientific Computing Research (ASCR); National Science Foundation (NSF)
Grant/Contract Number:
SC0012636; AC02-05CH11231; EAR-1033462
OSTI ID:
1603369
Alternate ID(s):
OSTI ID: 1568907
Journal Information:
Future Generations Computer Systems, Vol. 101, Issue C; Related Information: R. Ferreira da Silva, S. Callaghan, T. M. A. Do, G. Papadimitriou, and E. Deelman, “Measuring the Impact of Burst Buffers on Data-Intensive Scientific Workflows,” Future Generation Computer Systems, vol. 101, p. 208–220, 2019.; ISSN 0167-739X
Publisher:
ElsevierCopyright Statement
Country of Publication:
United States
Language:
English
Citation Metrics:
Cited by: 9 works
Citation information provided by
Web of Science

Figures / Tables (16)

Figure 1(p. 3)figure Figure 1
Figure 2(p. 4)figure Figure 2
Figure 3(p. 5)figure Figure 3
Figure 4(p. 6)figure Figure 4
Figure 5(p. 6)figure Figure 5
Figure 6(p. 7)figure Figure 6
Figure 7(p. 8)figure Figure 7
Figure 8(p. 8)figure Figure 8
Figure 9(p. 9)figure Figure 9
Figure 10(p. 10)figure Figure 10
Figure 11(p. 10)figure Figure 11
Figure 12(p. 11)figure Figure 12
Figure 13(p. 11)figure Figure 13
Figure 14(p. 11)figure Figure 14
Figure 15(p. 11)figure Figure 15
Table 1(p. 12)table Table 1

Similar Records

Related Subjects

42 ENGINEERING
Scientific Workflows
Burst Buffers
High-Performance Computing
In Transit Processing