We are currently preparing a new setup for the hosting of www.paessler.com. Because visitors to our website are coming from all over the world we decided to host CSS, images and media files on a global content delivery network in order to provide a swift website experience for all visitors.
The advantage of using a CDN (Content Delivery Network, see wikipedia
) is that website visitors will be served media files from the closest "edge-server" of the CDN and not from our own servers in the Rackspace data center in Dallas TX (the dynamic HTML would still be served from Dallas, though).
We have been using Amazon CloudFront CDN to deliver our software downloads for almost a year with excellent results. To find the best CDN provider for our upcoming website we set up accounts on selected CDN sites and selected hosting platforms. Then we monitored the performance from six locations around the globe. The contestants are:
For this test the PRTG Network Monitor
monitoring probes of our cloud monitoring system www.cloudclimate.com
requested a 64 kbyte file every 30 seconds between Aug 8th and today. We used probes located at GoGrid.com (San Francisco CA), Softlayer (Dallas TX), Newservers.com (Miami FL), Amazon EC2 (US and EU data center) and Flexiscale (UK). Here are the results:
Average Request Times
Let's remember that our goal is to provide fast downloads to people everywhere. So we would like to see short bars which should have the same short height for all locations.
Note: We haven been talking to people at Amazon about the elevated Cloudfront values for EC2 US (blue bar) and they were able to fix the routing so that the latest values are around the 60ms area. The same process is currently ongoing for Newservers (purple bar). So actually we can consider all CloudFront values to be excellent.
Request Time Observations
CloudFront CDN and Cloud Files CDN lead the pack, closely followed by CloudLayer CDN. These three are also the most expensive solutions, but they are still much cheaper than our Rackspace bandwidth and much faster. From a performance perspective SimpleCDN is not better than "normal" hosting, but it is much cheaper, so it may be a good and cheap solution for very bandwidth hungry downloads (e.g. ISO images, virtual machines) that don't need the latest bit of performance. Google's AppEngine was the surprise for me in this test. It is not a CDN and it is slower than the CDNs, but request times are quite evenly distributed across the US and EU. As we all know Google stores all data in a distributed database and this shows. I'd rate them the price/performance-ratio leader. Storage system S3, cloud hoster Joyent and our own web servers in Dallas all show request times that corelate to the geographical distance between server and probes.
Ratio "Average Request Times" vs. "95th Percentile"
The ratio between the average and the 95th percentile gives us an estimate how consistent the request times were. A higher ratio tells us that request times were more volatile. Of course we would like to have constant times (so many small bars on the graph are better).
From this perspective the hosting servers and S3 are better than the CDNs. But even when CloudFront needs twice as long as the average (+100%) it is still twice as fast as an average S3 request. Cloudfront is not only faster than Cloud Files, it also delivers more consistently, too.
Here are the average request times for all contestants from all six locations:
We will stay with Amazon CloudFront. It delivers our files in only 25% of the time that our own webserver in Dallas would need (or you could say: our webpages will load 4 times faster). CloudFront delivers files a little more consistent than Cloud Files, making Cloud Files a close runner-up. Softlayer's CDN also performs quite well and we rate it third in our test. An impressive fourth place goes to AppEngine. SimpleCDN may be cheap, but it is not fast enough for us. In our search for a fast way to quickly deliver files to our website visitors regardless of where they are all other contestants are no options.
What about Asia and the rest of the world? We also did some monitoring from Panama and Singapore and results showed quite similar patterns. But our probes there are not reliable enough to be included in the official test.