Abstract
Container technology has gained great popularity in cloud environment since containers provide near-native performance and are lighter and less expensive than traditional virtual machines. However, starting up a non-local container, whose image is unavailable locally, is time-consuming. The reason is that pulling an image from a remote registry requires a long time. In this paper, we experimentally find that a lot of redundant data exists among image layers when we pull images from the registry. This redundant data causes additional pull time and makes the startup of a non-local container slower. To minimize the amount of pulled data while providing intact images, we propose a container deployment framework, BED, based on block-level deduplication. To be specific, BED stores an image layer as numerous data blocks and a fingerprint list which is generated based on these data blocks. When BED needs to pull an image, it pulls the fingerprint lists of the image, deduplicates these fingerprint lists, and pulls the data blocks non-existing locally from the registry. Based on the local and pulled blocks, BED reconstructs the image layers for a container. Experiments show that compared with original Docker, BED reduces the time of pulling images by 35% on average and saves about 48% data transmission in the network.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Docker. https://www.docker.com/
Dragonfly. https://d7y.io/en-us/
Kubernetes. https://kubernetes.io/
Linux Cgroups. https://www.kernel.org/doc/Documentation/cgroup-v1/
Linux Namespaces. https://www.kernel.org/doc/Documentation/namespaces/
Linux Traffic Control. https://www.lartc.org/
Open Container Initiative. https://www.opencontainers.org/
Anwar, A., et al.: Improving docker registry design based on production workload analysis. In: Proceedings of FAST, pp. 265–278. USENIX (2018)
Bernstein, D.: Containers and cloud: from LXC to docker to kubernetes. IEEE Cloud Comput. 1(3), 81–84 (2014)
Constantinescu, C., Glider, J.S., Chambliss, D.D.: Mixing deduplication and compression on active data sets. In: Proceedings of DCC, pp. 393–402. IEEE (2011)
Darrous, J., Ibrahim, S., Zhou, A.C., Pérez, C.: Nitro: network-aware virtual machine image management in geo-distributed clouds. In: Proceedings of CCGRID, pp. 553–562. IEEE (2018)
Harter, T., Salmon, B., Liu, R., Arpaci-Dusseau, A.C., Arpaci-Dusseau, R.H.: Slacker: fast distribution with lazy docker containers. In: Proceedings of FAST, pp. 181–195. USENIX (2016)
Herbein, S., et al.: Resource management for running HPC applications in container clouds. In: Kunkel, J.M., Balaji, P., Dongarra, J. (eds.) ISC High Performance 2016. LNCS, vol. 9697, pp. 261–278. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41321-1_14
Huang, Z., Wu, S., Jiang, S., Jin, H.: FastBuild: accelerating docker image building for efficient development and deployment of container. In: Proceedings of MSST, pp. 28–37. IEEE (2019)
Keelveedhi, S., Bellare, M., Ristenpart, T.: DupLESS: server-aided encryption for deduplicated storage. In: Proceedings of the 22th USENIX Security Symposium, pp. 179–194. USENIX (2013)
Lin, C., Cao, Q., Huang, J., Yao, J., Li, X., Xie, C.: HPDV: a highly parallel deduplication cluster for virtual machine images. In: Proceedings of CCGRID, pp. 472–481. IEEE (2018)
Lin, X., Hibler, M., Eide, E., Ricci, R.: Using deduplicating storage for efficient disk image deployment. EAI Endorsed Trans. Scalable Inf. Syst. 2(6), e1 (2015)
Mandal, S., et al.: Using hints to improve inline block-layer deduplication. In: Proceedings of FAST, pp. 315–322. USENIX (2016)
Merkel, D.: Docker: lightweight Linux containers for consistent development and deployment. Linux J. 2014(239), 2–6 (2014)
Muthitacharoen, A., Chen, B., Mazières, D.: A low-bandwidth network file system. In: Proceedings of SOSP, pp. 174–187. ACM (2001)
Puzio, P., Molva, R., Önen, M., Loureiro, S.: ClouDedup: secure deduplication with encrypted data for cloud storage. In: Proceedings of CloudCom, pp. 363–370. IEEE (2013)
Quinlan, S., Dorward, S.: Venti: a new approach to archival storage. In: Proceedings of FAST, pp. 89–101. USENIX (2002)
Süß, T., Kaya, T., Mäsker, M., Brinkmann, A.: Deduplication analyses of multimedia system images. In: Proceedings of HotEdge. USENIX (2018)
Thalheim, J., Bhatotia, P., Fonseca, P., Kasikci, B.: CNTR: lightweight OS containers. In: Proceedings of ATC, pp. 199–212. USENIX (2018)
Wang, K., Yang, Y., Li, Y., Luo, H., Ma, L.: FID: a faster image distribution system for docker platform. In: Proceedings of FAS*W@SASO/ICCAC, pp. 191–198. IEEE (2017)
Xia, W., et al.: FastCDC: a fast and efficient content-defined chunking approach for data deduplication. In: Proceedings of ATC, pp. 101–114. USENIX (2016)
Zhu, B., Li, K., Patterson, R.H.: Avoiding the disk bottleneck in the data domain deduplication file system. In: Proceedings of FAST, pp. 269–282. USENIX (2008)
Acknowledgment
We thank Shadi Ibrahim for his valuable advice. This work is supported by National Key Research and Development Program under grant 2016YFB1000501, National Science Foundation of China under grants No.61872155 and 61732010.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, S., Wu, S., Fan, H., Zou, D., Jin, H. (2020). BED: A Block-Level Deduplication-Based Container Deployment Framework. In: Yu, Z., Becker, C., Xing, G. (eds) Green, Pervasive, and Cloud Computing. GPC 2020. Lecture Notes in Computer Science(), vol 12398. Springer, Cham. https://doi.org/10.1007/978-3-030-64243-3_38
Download citation
DOI: https://doi.org/10.1007/978-3-030-64243-3_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-64242-6
Online ISBN: 978-3-030-64243-3
eBook Packages: Computer ScienceComputer Science (R0)