High traffic websites like CNN, Yahoo!, Adobe, Rediff, etc. use a Content Delivery Network (like Akamai) so end users have to spend less time waiting for the web page to load on their screens.
Why You Need a Content Delivery Network?
Let’s take a real example: This website is hosted on a web server that’s located in some part of America. Now if we have a visitor from Singapore, the page loading time for him will be relatively high because of the geographic distance between Singapore and America. Had there been a mirror server in either India or Australia, the page would load much faster for that visitor from Singapore.
Now a content delivery network has servers across the world and they automatically determine the fastest (or the shortest) route between the server hosting the site and the end-user. So your page will be served from the server in Australia to a visitor in Singapore and from America for a visitor in UK. Of course there are other advantages but this example should give you a good idea of why we need a Content Delivery Network.
I am not as big as CNN. Can I still afford a Content Delivery Network
Absolutely. If you are using Amazon S3 for hosting images, you can consider using their their new CloudFront service for content delivery. Like Amazon S3, CloudFront is inexpensive (you pay for what you use) and gives all the benefits of a typical CDN service.
Location of Amazon Data Centers World-wide. Credit Werner Vogels
When you upload files to Amazon S3 and configure them with CloudFront, a copy of S3 files will be cached in various Amazon servers that are spread across important cities in US, Europe and Asia. So a visitor from India will be served content by the server located in Asia and so on – see illustration above for location of all Amazon servers.
How to Setup Amazon S3 with CloudFront as your Content Delivery Network
This is a simple step-by-step tutorial on how to serve your images and other media faster to end-users using the Amazon S3 + CloudFront Content Delivery Network:
Step 1: Get the S3 Fox add-on for Firefox and log-in to your Amazon S3 account. Now right-click your S3 bucket and select "Manage Distributions."
Step 2: Next we’ll associate a CNAME (some easy to remember sub-domain name) with our S3 bucket so it become easy to link to the files from our web pages. In this example, I use the web address "cache.labnol.org" and then click Create Distribution.
Step 3: The status will change from "In Progress" to "Deployed". Now copy the resource URL to the clipboard which is of the format xyz.cloudfront.net
Step 4. Log in to your account on your web hosting service’s website, and go to the DNS management page. Create a new CNAME record as shown in the screenshot. The CNAME name is the same sub-domain you created above while the CNAME value is your cloudfront.net URL generated by Amazon.
It may take a while (5 minutes?) for the domain change to propagate across different DNS servers. You may use online nslookup to confirm that the changes have been correctly propagated.
And you are done. Simply replace the image URLs in your web templates with the new sub-domain (in our case, cache.labnol.org) and your done.
Should You Use Amazon S3 or Amazon CloudFront
Amazon S3 is the place where you store content (your files) while CloudFront ensures that you files stored in S3 buckets get served faster.
CloudFront does this by copying files from S3 buckets to different edge locations that in turn serve content to end-users.
Edge Locations are spread across the world and content is always served from an Amazon server that is closest to the user’s location thus improving the page loading time as it means fewer network hops.
Statistics from YDN says that 80-90% of the end-user response time is spent downloading all the components in the page: images, stylesheets, scripts, Flash, etc. We can therefore disperse all this static content using a content delivery network an improve the speed of our web site..
In the case of S3, you were only paying for data served from S3 to the end user but in case of CloudFront, you also need to pay for data transferred from S3 to these edge locations.
Related: How to Lower your Amazon S3 Bill
Find this article at: http://www.labnol.org/internet/setup-content-delivery-network-with-amazon-s3-cloudfront/5446/
web: http://www.labnol.org/ email: amit@labnol.org
Reader Comments
Amit, I am one of the developers at Bucket Explorer, and wanted to let your readers know that we have released an updated version of Bucket Explorer with support for managing CloudFront. A beta can be downloaded from this link
Written by Saurabh on 11.18.08
this is good but I have some questions:
1. why not simply link the files (like images) instead of changing the cname?
2. why pay for amazon, since my host Media Temple has all the bandwidth I need? I have now 10K pageviews/day, not much but at which point starts to worth the price?
3. Does this save server CPU too?
4. Are you using Amazon S3? Or this cloud service?
thanks
Written by Gustavo on 11.18.08
Amazon’s new service does not “give all the benefits of a typical CDN service.” The new Amazon service is limited it what it can offer, which is fine for the right customer, but it will not work for customers who expect a CDN to provide support for streaming, live broadcasting, SLA, credit for outages, human phone support and a host of other services that most CDNs offer.
Written by Dan Rayburn on 11.18.08
Amit,
Is Japan or Hong Kong a better location (than US) to serve users in India?
Written by Shashi on 11.18.08
thank you so much for this! I was trying to use the perl script that aws suggests and this method is 100 times easier than that. thanks again!
Written by matt on 11.18.08
Your post made it a bit easy for me to add a CDN for SnappyFingers. Thanks!
In time, inexperienced users are bound to get confused when they decide to modify files that are cached by the CDN. Even a browser refresh will not download the latest version of the media file from the CDN. (Default expiry time in the CDN is 24hrs). It might be worthwhile to do a future post on versioning of files that will be served by the CDN.
Written by Chirayu on 11.19.08
Thanks for this post. It helped me get set up with distributions and CNAME’s right away.
One thing that caused a small issue was setting up the CNAME value. The article says to just use your CloudFront.net URL generated by Amazon as the CNAME value.
Upon attemping to use the full URL, http:// included, my web host denied it because of the link portion. Once I removed that, it worked fine.
Maybe this is assumed, but perhaps some other readers may appreciate the clarification.
Thanks!
Written by Matthom on 11.19.08
Can I point a whole domain like example.com to a distribution or does it have to be a subdomain? Thanks for the informative post!
Written by Mike on 11.21.08
A couple things worth mentioning,
Wait a week for the domain to propagate fully before going to production. On our site, we had customers calling in from remote areas (Alaska, Rural Puerto Rico) who claimed they couldn’t see our pictures. Found out it takes about a week for domain name changes to fully propagate.
Also, I’ve used S3 Fox (and still do for small fast uploads to S3) but for any heavy lifting I use bucket explorer. We send up a few hundred pictures up every night and with bucket explorers command line (bucket commander), and a versioning script, it makes deployment a cinch.
Written by bill h on 11.25.08
I think there is one start point you missed - you also need to signup cloudfront before using any of these suggestions. I spent some time in debugging those errors before knowing this.
BTW, thanks for the great article!
Written by Aditya Kothadiya on 01.09.09