Orchestrate a Thumbnailer efficiently and effectively with the Managed Kubernetes Service
OVHcloud and Todocoleccion
30 million
lots
200 million
stored images
625,000
visitors a day
The background
Todocoleccion is the largest online community for buying and auctioning antiques, art, books and all kinds of collectibles.
For 25 years, Todocoleccion has been offering its users a storefront where they can buy or sell goods.
With more than 30 million lots for sale in over 2,000 categories, the company offers a catalogue “full of emotions, feelings and story-filled memories. Stories belonging to a community of small groups with a variety of cultural interests that make the world of collecting so special”.
To enable this community to buy and sell these items, Todocoleccion provides all the tools needed to manage the life cycle of a listing according to their buyers or sellers’ needs. Sellers have tools for mass lot management, mass import, and aftersales support management, while buyers have tools that allow them to communicate with sellers, bid, and make offers on the listings they want.
Pioneers in online commerce
As a pioneer in online commerce, Todocoleccion has faced many challenges over the past 25 years throughout its growth and expansion. In its initial version, the website was static and the lots for sale were published directly in HTML. Soon, the company underwent a restructuring with the addition of a new partner, aiming to revolutionise the collection and auction industry. The company now employs more than 50 people worldwide and has a hybrid infrastructure distributed between Bare Metal dedicated servers, Hosted Private Cloud and Public Cloud. With its website and mobile application, Todocoleccion receives over 10 million visitors viewing 100 million pages each month, with transactions totalling more than €65 million each year, 80% of which take place in Spain.
The challenge
An infrastructure for B2B, B2C and C2C transactions
Todocoleccion is a place for many communities of completely different sizes, with very specific and particular cultural interests. To provide each of these groups with the security and visibility they need for their online businesses, the company must have a highly available and reliable infrastructure. What’s more, Todocoleccion has a complex stack composed of multiple open-source tools. One of their biggest challenges is to keep their uptime to as close to 100% as possible.
“The range of physical and virtual solutions meant we turned to OVHcloud to improve the performance and scalability of our infrastructure, without having to maintain our own datacentre or use too many resources.”
Joaquín Valdés, CEO, Todocolección
A huge number of images to resize
To date, the company has a database of more than 200 million images of both sold and unsold products, and each of these images can be viewed in different resolutions, sizes and orientations or with different filters. In total, these images take up 14 TB in Object Storage. Todocoleccion therefore needs a highly available tool for creating thumbnails, at a controlled cost. To satisfy its customers, Todocoleccion has always relied on speed and the lowest latency possible. The company wanted to allow its users to access its website within a page response time of 200 ms whenever possible, with the optimal latency for a website being 100 ms.
A need for security and compliance
The startup industry in Malaga is very diverse. OVHcloud is strongly linked to this scene, as it offers scalable solutions at a controlled cost – ideal for startups and new developments. Since Todocoleccion is based in Europe and offers online shopping, the company needs a provider that meets European requirements in terms of security and data protection for its users.
“One of the more cultural reasons is that OVHcloud is a European project, with its headquarters in France and many subsidiaries around the world. OVHcloud generates jobs, wealth and, above all, is compliant with the regulations and laws in force.”
Joaquín Valdés, CEO, Todocolección
To meet their needs for scalability, high availability and security, all while managing the cost of their infrastructure, the company opted to migrate to OVHcloud.
What is a Thumbnailer?
A Thumbnailer is a tool/programme for generating thumbnails, i.e. reduced-size images, for a website. By using an image miniaturisation tool properly, you can reduce latency time for websites and optimise the visitor experience, among other features.
The user gives the Thumbnailer a file and a target size in pixels, and the programme creates a thumbnail that matches these settings.
What is the difference between Kubernetes and Docker?
Kubernetes and Docker work together – they’re complementary. To understand this complementarity, we need to understand containerisation: a method of partitioning operating systems (OS). This allows you to run environments that are completely isolated from each other but share the same “kernel” of the host machine.
Docker is a type of containerisation technology. The binary images, libraries and, in general, all the dependencies that an application needs in order to run, are placed in a “box”: a container kept totally independent from other containers sharing the same “kernel” as well as from the underlying infrastructure. Since these containers can be infinitely duplicated, managing them can quickly become a challenge. This is where Kubernetes comes in. Kubernetes is a container orchestrator. You can use it to manage the entire lifecycle of the Docker containers as soon as they are created. Creating and duplicating these Docker containers becomes simpler, as is distributing them across different infrastructures to automate their management more easily. Kubernetes allows developers to focus on their code without having to worry about the underlying layers and virtual machine (VM) management.
The solution
Migrating the infrastructure to OVHcloud
In July 2019, Todocoleccion chose to migrate part of their infrastructure to OVHcloud in order to reduce costs while maintaining close to 100% availability, even during the migration. The company chose OVHcloud as its new provider thanks to its wide range of physical and virtual solutions that allow it to improve its performance and scale its infrastructure, without having to manage its own On Premises datacentre – all while avoiding wasting resources. Since the complexity of its stack requires a solution suitable for its technical requirements, Todocoleccion employs a hybrid cloud solution in which Bare Metal Dedicated Servers, Hosted Private Cloud and Public Cloud are used to meet its various technical and specific needs.
“In addition to controlling our costs, we needed high availability, good response time and security measures that allow us to provide our services without interruption, or with a quick incident response and resolution in the event of interruption.”
Joaquín Valdés, CEO, Todocolección
Using Managed Kubernetes Service with a Thumbnailer
With more than 30 million lots to store and display, Todocoleccion needs an infrastructure that can generate, store and manage the thumbnails associated with the different products that are sold and auctioned on its website. So in October 2020, the company decided to launch an image miniaturisation tool to display listings on its website. They use their Managed Kubernetes cluster to operate their Thumbnail server and optimise the resources associated with creating thumbnails for the many lots available online, while experiencing as little latency as possible. Despite Todocoleccion’s unfamiliarity with Docker, its IT department quickly realised that Managed Kubernetes was the perfect solution for hosting their Thumbnailer.
“Although implementing this project seemed complex at first, we are delighted to have Adrian, who has been a true OVHcloud evangelist, helping us use our managed Kubernetes cluster.
The support provided to us has lived up to our expectations!”Joaquín Valdés, CEO, Todocolección
Increasing infrastructure reliability with automation
Todocoleccion also wanted to automate the lifecycle of its code deployed on Kubernetes as much as possible. By automating the way Docker images are generated and published to the Docker Registry, the company reduces points of potential errors by minimising the areas where human intervention is required in the deployment.
This thumbnail creation tool consists of N pods that resize and filter the images imported to the Todocoleccion website.
Deploying the Thumbnailer on Kubernetes consists of 8 to 50 pods created and deleted according to a Horizonal Pod Autoscaler (HPA) that automatically adds pods when the CPU reaches 80% of its capacity, and removes them when the CPU quota drops back down to 50%. Each pod runs the code for the Thumbnailer, which consists of a NodeJS application that communicates with OVHcloud Object Storage and then transforms the original image to send it as static content to the CDN using a C library.
Finally, Todocoleccion’s use of Kubernetes makes it easy to manage a clearly-defined software lifecycle. The company uses a GitLab-based CI/CD pipeline, which tests the JavaScript, compiles it, and generates a Docker image. This image is then applied to a Managed Kubernetes cluster for uploading. In addition to managing its code cycle using this CI/CD pipeline, Todocoleccion also deploys tools for application tracking directly in its Managed Kubernetes cluster, with tools like Prometheus and Grafana. Thanks to these measures, Todocoleccion can monitor each update of the Thumbnailer code and ensure that it is carried out smoothly. In the event of failures, the IT team can intervene at any point between setup and production. Todocoleccion’s Thumbnailer allows it to deliver more than 46 million image requests for over 625,000 users a day – a great example of using a Managed Kubernetes Cluster for scalability!
The result
After migrating their infrastructure to OVHcloud Public Cloud instances in July 2019, Todocoleccion reduced their website’s response time by nearly 80 ms. The Thumbnailer tool, which went into production in October 2020, can deliver 120 to 250 requests per second. In normal times, resource usage remains similar to the company’s previous implementation – but a major advantage is that Todocoleccion can resize its cluster by increasing the number of worker nodes as needed. This way, it can guarantee continued availability, even in periods of high traffic, by enabling auto-scaling for Kubernetes node-pools.
“OVHcloud allows us to get the same results as our previous provider, or even better results, at a fraction of the cost.”
Joaquín Valdés, CEO, Todocolección
Todocoleccion also wants to develop the use of mobile devices, which already account for 55% of the company’s current traffic and turnover, as well as continue to expand its international presence.
