This is a money-saving tip for web publishers who use Amazon S3 for hosting images and other static content like CSS, JavaScript files, etc.
Since Amazon S3 is a "pay as you use" storage service, your S3 bill is always directly proportional to the bandwidth that your sites consume.
How Browsers Interact with Amazon S3
When a visitor comes to your site the first time, the static images are downloaded from Amazon S3 servers and get saved inside his browser’s cache.
Now if that same person visits your site again sometime in future, his browser will make another GET request to Amazon S3 asking for a fresh copy of the web images.
Since the images stored on Amazon S3 haven’t changed since his last visit, Amazon servers will return a 304 Not Modified header response indicating that there’s no need to download images again.
So far so good. That 304 response prevented the visitor’s browser from downloading the same data again (thus saving you money) but there’s another problem – Amazon S3 also charges you for every GET request so each time a browser asks Amazon if images have changed since last visit, that question itself adds to your bill even if the answer is "no".
How to Reduce Your Amazon S3 Bill
While the cost for GET requests is small (just 1¢ per 10,000 requests), they can quickly add-up if you have have a popular site or if your website design uses too many images. For instance, every page on www.labnol.org has about 25 static images served from S3.
To control this cost, you need a mechanism that will prevent browsers from sending the GET request if the file already exists in their cache. This can be easily done by setting appropriate Cache-Control and Expires headers at the time of uploading the files on to Amazon S3.
Cache-Control is like instructing the browser whether to make any requests to Amazon S3 or not before a given period. So if you set Cache-Control max-age=864000 for your S3 images, the web browsers will not request that file from S3 storage until the next 10 days (3600*24*10 sec).
Other than saving money, your site will also load relatively faster because the visitor’s browser will reuse images, logos and other static files from the cache without making any new request to Amazon S3.
BitRhymes, developers of the popular Sketch Me app for MySpace, saw their Amazon S3 bill dip by 40% after they implemented cached headers for images.
Implement Caching for Amazon S3 Files
To set the appropriate Cache-Control headers for files hosted on Amazon S3, you can either use the Bucket Explorer client (cost $50) or upload files manually via this PHP script written by Lalit Patel who is also the inspiration behind this article.
If you are worried about setting Cache headers for JavaScript and CSS files as they may change frequently (especially when you are in the midst of a site re-design), Lalit shares a very simple workaround – just append a version number after the file name like main.js?v=2.
Before: <link href="http://files.labnol.org/main.css?v=2" ../>
After: <link href="http://files.labnol.org/main.css?v=3" ../>
Change the version from 2 to 3 and visitors browser will make a fresh GET request to Amazon S3 for the latest version of the S3 file.
Find this article at: http://www.labnol.org/internet/lower-amazon-s3-bill-improve-website-loading-time/5193/
Reader Comments
Putting a querystring on a URL means that it won’t ever be cached by most proxy servers, Opera or Safari. It’s a bad idea – best to change the filename of your CSS/JS/Image files when they change content.
Written by Jason on 11.03.08
Be careful using that ?v=N trick to get fresh cash. IE can be notorious for not bothering to re-request if a variable has changed. Sounds stupid, is stupid, but is also a problem I’ve run into.
Written by Jeff on 11.03.08
How good is browser support for this header?
Written by James Moss on 11.03.08
I’m not sure (as the files are hosted on a different server — S3) but you can try a .htaccess directive on your site root instead of worrying about the files individually.
Header set Expires “Sat, 01 Jan 2050 01:01:01 GMT”
Written by Brajeshwar on 11.04.08
Just came to know about Space Block ( link . It is FREE and powerful UI for S3 which allows cache-control edits.
Written by Lalit Patel on 11.04.08
Hello Amit, Taking your previous post as guidance I managed to put the images on Amazon s3, I deal with multiple images, is there anyway I can generate the embed code, something like what rightload does for link you
Written by Jeewan on 11.04.08
I implemented this cache-control setting on an IIS based application (not using S3) a few years back and the results were incredible. Here are the results that I saw after adding the header value to the images on our site:
===========================================================
TOTAL IMAGE REQUESTS (BEFORE) 975,092
TOTAL IMAGE REQUESTS (AFTER) 488,897
SAVINGS 50%
TOTAL IMAGE BYTES (BEFORE) 362,323,481
TOTAL IMAGE BYTES (AFTER) 280,941,834
SAVINGS 23%
TOTAL TIME TAKEN (BEFORE) 154,139,194
TOTAL TIME TAKEN (AFTER) 70,213,614
SAVINGS 55%
===========================================================
I just purcahsed Bucket Explorer and I will be using that to set the cache-control setting for our S3 account. I am not too worried about the financial savings since S3 is so inexpensive – to me its more important to reduce the number of uncessary requests from the browser so it can focus on loading dynamic content from our site.
Written by marc on 11.05.08