{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,7]],"date-time":"2024-08-07T23:37:12Z","timestamp":1723073832643},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2020,8]]},"abstract":"Monarch is a globally-distributed in-memory time series database system in Google. Monarch runs as a multi-tenant service and is used mostly to monitor the availability, correctness, performance, load, and other aspects of billion-user-scale applications and systems at Google. Every second, the system ingests terabytes of time series data into memory and serves millions of queries. Monarch has a regionalized architecture for reliability and scalability, and global query and configuration planes that integrate the regions into a unified system. On top of its distributed architecture, Monarch has flexible configuration, an expressive relational data model, and powerful queries. This paper describes the structure of the system and the novel mechanisms that achieve a reliable and flexible unified system on a regionalized distributed architecture. We also share important lessons learned from a decade's experience of developing and running Monarch as a service in Google.<\/jats:p>","DOI":"10.14778\/3181-3194","type":"journal-article","created":{"date-parts":[[2020,9,14]],"date-time":"2020-09-14T22:46:35Z","timestamp":1600123595000},"page":"3181-3194","source":"Crossref","is-referenced-by-count":14,"title":["Monarch"],"prefix":"10.14778","volume":"13","author":[{"given":"Colin","family":"Adams","sequence":"first","affiliation":[{"name":"Google LLC"}]},{"given":"Luis","family":"Alonso","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Benjamin","family":"Atkin","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"John","family":"Banning","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Sumeer","family":"Bhola","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Rick","family":"Buskens","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Ming","family":"Chen","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Xi","family":"Chen","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Yoo","family":"Chung","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Qin","family":"Jia","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Nick","family":"Sakharov","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"George","family":"Talbot","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Adam","family":"Tart","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Nick","family":"Taylor","sequence":"additional","affiliation":[{"name":"Google LLC"}]}],"member":"320","published-online":{"date-parts":[[2020,9,14]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.5555\/3026877.3026934"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.is.2015.04.007"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.5555\/2930583.2930587"},{"key":"e_1_2_1_5_1","unstructured":"A. Bader O. Kopp and M. Falkenthal. Survey and comparison of open source time series databases. In B. Mitschang D. Nicklas F. Leymann H. Sch\u00f6ning M. Herschel J. Teubner T. H\u00e4rder O. Kopp and M. Wieland editors Datenbanksysteme f\u00fcr Business Technologie und Web (BTW 2017) - Workshopband pages 249--268. Gesellschaft f\u00fcr Informatik e.V. 2017. A. Bader O. Kopp and M. Falkenthal. Survey and comparison of open source time series databases. In B. Mitschang D. Nicklas F. Leymann H. Sch\u00f6ning M. Herschel J. Teubner T. H\u00e4rder O. Kopp and M. Wieland editors Datenbanksysteme f\u00fcr Business Technologie und Web (BTW 2017) - Workshopband pages 249--268. Gesellschaft f\u00fcr Informatik e.V. 2017."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3264903"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/362686.362692"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2012.37"},{"key":"e_1_2_1_9_1","unstructured":"E. Brewer. Spanner truetime and the cap theorem. Technical report 2017. E. Brewer. Spanner truetime and the cap theorem. Technical report 2017."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1365815.1365816"},{"key":"e_1_2_1_11_1","first-page":"124","volume-title":"2004 IEEE International Conference on Mobile Ad-hoc and Sensor Systems","author":"Chen H.","year":"2004"},{"key":"e_1_2_1_12_1","unstructured":"B. S. Chris Larsen. OpenTSDB - a distributed scalable monitoring system. http:\/\/opentsdb.net. B. S. Chris Larsen. OpenTSDB - a distributed scalable monitoring system. http:\/\/opentsdb.net."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2491245"},{"key":"e_1_2_1_14_1","unstructured":"R. Cox. Regular expression matching with a trigram index 2012. https:\/\/swtch.com\/~rsc\/regexp\/regexp4.html. R. Cox. Regular expression matching with a trigram index 2012. https:\/\/swtch.com\/~rsc\/regexp\/regexp4.html."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-28534-9_16"},{"key":"e_1_2_1_16_1","unstructured":"J. Dixon. Monitoring with Graphite: Tracking Dynamic Host and Application Metrics at Scale. O'Reilly Media 1 edition March 2017. J. Dixon. Monitoring with Graphite: Tracking Dynamic Host and Application Metrics at Scale. O'Reilly Media 1 edition March 2017."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2379776.2379788"},{"key":"e_1_2_1_18_1","unstructured":"A. Fikes. Storage architecture and challenges. https:\/\/cloud.google.com\/files\/storage_architecture_and_challenges.pdf. A. Fikes. Storage architecture and challenges. https:\/\/cloud.google.com\/files\/storage_architecture_and_challenges.pdf."},{"key":"e_1_2_1_19_1","unstructured":"L. George. HBase: The Definitive Guide. O'Reilly Media 1 edition 2011. L. George. HBase: The Definitive Guide. O'Reilly Media 1 edition 2011."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/945445.945450"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2011.389"},{"key":"e_1_2_1_22_1","unstructured":"Google. Snappy --- a fast compressor\/decompressor. https:\/\/github.com\/google\/snappy. Google. Snappy --- a fast compressor\/decompressor. https:\/\/github.com\/google\/snappy."},{"key":"e_1_2_1_23_1","unstructured":"K. W.\n \n Hipel\n and \n \n \n A. I.\n \n McLeod\n .\n \n \n \n Time series modelling of water resources and environmental systems volume \n 45\n of \n Developments in Water Science\n . \n Elsevier 1994\n . K. W. Hipel and A. I. McLeod. Time series modelling of water resources and environmental systems volume 45 of Developments in Water Science. Elsevier 1994."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.5555\/2813767.2813814"},{"key":"e_1_2_1_25_1","unstructured":"F. Inc. Beringei: a high performance in memory time series storage engine 2016. https:\/\/github.com\/facebookarchive\/beringei. F. Inc. Beringei: a high performance in memory time series storage engine 2016. https:\/\/github.com\/facebookarchive\/beringei."},{"key":"e_1_2_1_26_1","unstructured":"G. Inc. gRPC: Bidirectional streaming RPC 2017. https:\/\/grpc.io\/docs\/guides\/concepts\/. G. Inc. gRPC: Bidirectional streaming RPC 2017. https:\/\/grpc.io\/docs\/guides\/concepts\/."},{"key":"e_1_2_1_27_1","unstructured":"InfluxData. InfluxDB --- open source time series metrics and analytics database. http:\/\/influxdata.com. InfluxData. InfluxDB --- open source time series metrics and analytics database. http:\/\/influxdata.com."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.5555\/645919.672812"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/tkde.2017.2740932"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.14778\/3236187.3236215"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.14778\/2732951.2732953"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.5555\/3199517.3199519"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2003.1260811"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2005.01.025"},{"key":"e_1_2_1_35_1","volume-title":"Proceedings of the 8th Biennial Conference on Innovative Data Systems Research","author":"Meehan J.","year":"2017"},{"key":"e_1_2_1_36_1","volume-title":"CO","author":"Merchant A.","year":"2014"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.14778\/2824032.2824078"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2339530.2339576"},{"key":"e_1_2_1_39_1","unstructured":"F. Reinartz J. Volz and B. Rabenstein. Prometheus --- monitoring system & time series database. http:\/\/prometheus.io\/. F. Reinartz J. Volz and B. Rabenstein. Prometheus --- monitoring system & time series database. http:\/\/prometheus.io\/."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.14778\/3229863.3229871"},{"key":"e_1_2_1_41_1","volume-title":"Inc.","author":"Sigelman B. H.","year":"2010"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1504\/IJSNET.2006.012035"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.5555\/1182635.1164196"},{"key":"e_1_2_1_44_1","doi-asserted-by":"crossref","unstructured":"S. J. Taylor. Modelling Financial Time Series. World Scientific second edition 2007. S. J. Taylor. Modelling Financial Time Series. World Scientific second edition 2007.","DOI":"10.1142\/6578"},{"key":"e_1_2_1_45_1","unstructured":"The Linux man-pages project. cgroups --- Linux control groups. http:\/\/man7.org\/linux\/man-pages\/man7\/cgroups.7.html. The Linux man-pages project. cgroups --- Linux control groups. http:\/\/man7.org\/linux\/man-pages\/man7\/cgroups.7.html."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2741948.2741964"},{"key":"e_1_2_1_47_1","first-page":"107","volume-title":"Site Reliability Engineering","author":"Wilkinson J.","year":"2016"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/CloudCom.2012.6427510"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3181-3194","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T12:05:54Z","timestamp":1672229154000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3181-3194"}},"subtitle":["Google's planet-scale in-memory time series database"],"short-title":[],"issued":{"date-parts":[[2020,8]]},"references-count":47,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2020,8]]}},"alternative-id":["10.14778\/3181-3194"],"URL":"https:\/\/doi.org\/10.14778\/3181-3194","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2020,8]]}}}
