Mercado Libre Scales Its Business, Improves Reliability in Migrating 5,000 Databases to Amazon DynamoDB

Founded in Argentina in 1999, Mercado Libre features two main businesses: ecommerce and fintech. Within those realms, the company offers many services, including an online marketplace, a digital payment system, advertising, logistics, and credit offerings. Its underlying purpose is to democratize commerce, payments, and money to make a difference in the lives of millions of people in Latin America.

Mercado Libre had originally built its solutions on a virtual private cloud, with each internal team responsible for handling its own infrastructure, applications, and maintenance. Individual teams determined, for example, which types of databases to use. Although this was key for fast growth at first, as the number of teams grew and the traffic increased, the variety of technology stacks made it difficult to sustain growth. The company built its teams a new solution that included a proprietary software development kit and custom APIs. For NoSQL databases, instead of providing direct access to any solution, the company built a layer of abstraction so that all teams were speaking the same language. Initially, Mercado Libre built its key-value main database abstraction on top of a self-managed, open-source database engine, a solution that grew to 5,000 databases.

But over time, the company realized it wasn’t getting all the features it needed from the selected engine. Plus, managing over 30 clusters required frequent maintenance, resulting in timeouts, errors, and downtime incidents. In early 2019, Mercado Libre began its migration to Amazon DynamoDB. “With the reliability of Amazon DynamoDB, we can avoid the cost of downtime,” says Oscar Mullin, Mercado Libre’s director of IT and head of core services, site reliability engineering, and database administration. Using the common abstraction to its advantage, the company migrated 5,000 databases without requiring anyone in the organization to change a single line of code or alter the business road map.

Mercado Libre used Amazon DynamoDB alongside other AWS services that Mercado Libre was usingto run its mission- and business-critical applications, such asAmazon Elastic Compute Cloud(Amazon EC2), a web service that provides secure, resizable compute capacity in the cloud, andAmazon Simple Storage Service(Amazon S3), an object storage service that offers industry-leading scalability, data availability, security, and performance.

Using the combined power of AWS solutions, Mercado Libre was able to handle all of its scaling needs without impacting performance. Previously, the company spent significant time preparing for scaling events such as holidays, and scaling was complex. And as a result of unexpected events such as the COVID-19 pandemic, more people have turned to ecommerce and digital payments, which significantly increased traffic for Mercado Libre. As part of its mission of inclusion, Mercado Libre aimed to use AWS to extend its offerings to more small businesses. “Having our core applications using our abstraction based on Amazon DynamoDB was critical,” says Mullin. “Coping with that traffic increase while keeping the level of availability would have been extremely complex otherwise.” The automatic scaling capabilities of Amazon DynamoDB also proved invaluable when the company needed to downscale, helping it prevent unused capacity, reduce waste, and cut costs. “Unlike on our previous solutions, downscaling on Amazon DynamoDB is extremely simple,” says Mullin.

The multi–AWS Region architecture supported by Amazon DynamoDB also solves availability issues for Mercado Libre. Instead of having to transfer data found in separate tables, the company usesAmazon DynamoDB global tables,which replicate tables automatically across a company’s choice of Regions for multi-Region fault tolerance. This fully managed, multi-Region, multiactive database helps deliver fast, local read/write performance to millions of globally dispersed customers. “We have the ability to be multi-Region with a single click,” says Mullin. “And all our historical data moves to the other Regions automatically.” Changes to one table seamlessly propagate to the other Regions, saving time for the company. No matter where users are located, they can access data with low latency. In fact, the ninety-ninth percentile of combined latency of API and Amazon DynamoDB reads is just 6 ms for read requests. And even as requests reach a rate of 300 million per minute, latency doesn’t suffer. Plus, Mercado Libre no longer has to navigate complex processes just to get a daily backup because Amazon DynamoDB offers point-in-time recovery for continual, automatic backups.

Mercado Libre’s approach to services includes building interrelated services that provide its developers with transparent integrations that are as simple as possible. To achieve this integration with the key-value store, Mercado Libre turned toAmazon DynamoDB Streams,which captures a time-ordered sequence of item-level modifications in any Amazon DynamoDB table and stores the information in a log for 24 hours. Also, DynamoDB Streams works with Amazon DynamoDB in automatically removing irrelevant information from tables, all for no additional cost. Mercado Libre can use Amazon DynamoDB to delete these expired items using time-to-live expiration, which removes unwanted or irrelevant items in the background without consuming any write throughput. And by using DynamoDB Streams, the company can capture these deletions and back up or otherwise process data as it expires. “DynamoDB Streams is a powerful feature,” says Mullin. “Using its automated capabilities removes the need for us to do many tasks ourselves, a process that was complex and error prone.”

Although Mercado Libre originally selected Amazon DynamoDB as a way to off-load operational overhead and support massive scale, its developers are continuing to unlock new capabilities, driving the company’s ability to innovate and continue to grow in Latin America. “When we analyzed alternatives to our previous setup, for what we needed, Amazon DynamoDB was a clear winner for us,” says Mullin. “And we continue to get value from it by building new abstractions like secondary indexes or security capabilities.”