Understanding the latency benefits of multi-cloud webservice deployments

Zhe Wu, Harsha V. Madhyastha
Appears in: 
CCR April 2013

To minimize user-perceived latencies, webservices are often deployed across multiple geographically distributed data centers. The premise of our work is that webservices deployed across multiple cloud infrastructure services can serve users from more data centers than that possible when using a single cloud service, and hence, offer lower latencies to users. In this paper, we conduct a comprehensive measurement study to understand the potential latency benefits of deploying webservices across three popular cloud infrastructure services - Amazon EC2, Google Compute Engine (GCE), and Microsoft Azure. We estimate that, as compared to deployments on one of these cloud services, users in up to half the IP address prefixes can have their RTTs reduced by over 20% when a webservice is deployed across the three cloud services. When we dig deeper to understand these latency benefits, we make three significant observations. First, when webservices shift from single-cloud to multi-cloud deployments, a significant fraction of prefixes will see latency benefits simply by being served from a different data center in the same location. This is because routing inefficiencies that exist between a prefix and a nearby data center in one cloud service are absent on the path from the prefix to a nearby data center in a different cloud service. Second, despite the latency improvements that a large fraction of prefixes will perceive, users in several locations (e.g., Argentina and Israel) will continue to incur RTTs greater than 100ms even when webservices span three large-scale cloud services (EC2, GCE, and Azure). Finally, we see that harnessing the latency benefits offered by multi-cloud deployments is likely to be challenging in practice; our measurements show that the data center which offers the lowest latency to a prefix often fluctuates between different cloud services, thus necessitating replication of data.

Public Review By: 
Katerina Argyraki

One of the benefits of hosting a web service in the cloud is closer proximity to the end-user: as a cloud typically consists of multiple datacenters, located in different places, a webservice provider can serve each client from the physically-closest datacenter, hence reduce user-perceived latency. This paper now tells us that hosting a web service in multiple clouds can further improve this benefit: by switching from a single-cloud to a multi-cloud deployment, 20-50% of IP prefixes would reduce their latency to the closest datacenter by more than 20%. These numbers are based on measurements from 265 PlanetLab nodes spanning multiple countries and all continents (with the exception of Africa). The authors report two reasons for this improvement: (i) cloud provider A may have a datacenter in a particular region, whereas cloud provider B does not; (ii) the routing toward cloud provider A's datacenter in a particular region may be significantly worse than the routing toward cloud provider B's datacenter in the same region. Whether these findings push webservice providers toward multicloud deployments or motivate ISPs to improve their routing, the reviewers agree that they are useful and, to some extent, unexpected. Moreover, the reviewers welcome more work on how to deal with latency fluctuations and how to improve quality of service in regions that are poorly served by all cloud providers.