First results: Which CDN Provider serves you fastest?

First, a word of caution about the results

The Javascript-based test at CloudClimate.com runs inside the user's browser window. In order to trigger 10 downloads, we add a unique "cachebreaker" URL parameter to each download URL. This causes some of the CDN providers to pull the images from their origin for each of the 10 requests (instead of pulling it once and then delivering it from the "edge server"). This of course skews the results heavily. Especially Akamai CDN, Voxcast CDN,Azure CDN and RackSpaceCloud CDN seem to have this issue.

Average Request Times

The following chart shows the average request times for each provider. The average request time even the fastest provider is 425 ms. That’s a lot slower than the measurements we found in our test last August where we ran our requests out of professional data centers around the globe. For the average real-world surfer the differences between the various offerings are much smaller than CDN marketing people want to make us think... But of course we want to make webpages load as fast as possible. The Top 10 are:

• 1. GoGrid/Edgecast CDN
• 2. SoftLayer/Internap CDN
• 3. CacheFly CDN
• 4. Cloudfront CDN
• 5. MaxCDN
• 6. Amazon Cloud S3 US
• 7. Google Appspot
• 8. Terremark Cloud
• 9. Azure CDN
• 10. SimpleCDN

Clear winner is the GoGrid/Edgecast CDN which beats all others. Followed by a group of CDNs (Softlayer/Internap, CacheFly, CloudFront and MaxCDN). Rank 6 and 8 were a little surprise: Amazon S3 (US Region) and the Terremark Cloud are the only single-location contenders in the top 10 and – in this real life test – show a performance that is in the range of the (more expensive) CDN providers. For web site visitors from Europe and the US (most of our test participants) S3 is quite fast. Number 7, Google AppEngine, is hosted on Google’s distributed architecture and has one interesting side aspect: It is free for up to 1.3 Mio requests and 1 gigabyte per day. For web sites with little to medium traffic this can be a great way to speed up the website without paying extra. We miss the CDNs of Akamai, Voxcast and RackSpaceCloud in the Top 10, I already explained the reason above.

Conclusion

We are looking at the performance of CDNs and cloud servers from the perspective of real-live-users. When we compare these the results for CDNs with the results for cloud hosted servers we can say that, as a rule of thumb, CDNs are about twice as fast as cloud servers (and likely dedicated servers) when it comes to delivering assets. For users with high speed connections the advantage of the CDNs is stronger (look at the minimum values, most CDNs are below 40ms with their minimum values). We will continue to look at this topic in the next months, stay tuned for more test results....

The data in numbers

Where did users come from?

The users, who ran this test inside their browsers came from: