overstimulate

While I've been playing with S3 for many months now, I've yet to rack up more than a few penny's in costs. After reading how smugmug saved hundreds of thousands using S3, I decided to fix a problem I had.

Due to lack of sleep for both myself and Britt, we had been serving static content via mongrel. (gasp!) In the past 2 months mongrel has fielded over 24 million requests (serving 307GB). Having mongrel serve static content (the site's media/css/javascript) meant a slowdown in serving up the dynamic pages. Luckily the transition to S3 serving all of that content was quick and painless.

If you use zoneedit, choose edit zone, then the domain, aliases (CNAME), then add static as an alias for userscripts.s3.amazonaws.com

Step 1: setup a cname: a quick check of Virtual Hosting of Buckets showed that all one has to do is setup static.userscripts.org to be a cname/alias to static.s3.amazonaws.com

Step 2: setup s3: create a bucket on S3 with the name of the domain you just setup as an alias

Using S3Fox Organizer for Amazon, you click the new folder button and type the domain name, while you can use s3sync.rb, bitbucket (python), or ... I prefer using this extension so I can see what is happening. S3Fox's UI treats a bucket as a directory and treats buckets and keys as if you only access them via s3.amazonaws.com/bucketname/key not bucketname.s3.amazonaws.com/key or using the cname trick we setup in step 1 - don't let this confuse you as all the methods of getting to the data will still work.

Step 3: put your content on s3: pushing your data to S3 and make it world readable.

Since userscripts is a rails app I copied the public directory, deleted a few files (dispatch.*, robots.txt, ...), removed all the .svn directories, then used S3Fox to copy the files up. By default the files are private, so clicking properties on each folder (images, stylesheets, javascripts) and setting it to be readable by the public (recursively -- apply to subfolders)

Step 4: use your new domain, by now all of your javascript, css, and images should be accessible by prepending existing urls with static. After testing that everything is working, you have only to change your stylesheet links, javascript sources, and image tags.

Updating application.rhtml to point to the right css/javascript is easy, then a quick grep through the views for image tags completed the transistion.

Results: The above took about 15 minutes to complete. After committing the changes I asked a few friends to kick the tires. I would have a friend browse userscripts for a few minutes with the latest changes (using S3) or the previous svn revision (not using S3), then after a few minutes change back. The results were unanimous with S3 providing a noticeable speed up. Ahh success!

Sure we could have setup apache/lighttpd to serve static content, probably in a roughly equivalent amount of time. I'm currently planing to copy the userscripts themselves to S3. My reason - peace of mind - if something were to happen to our server, having the content still available while the backups are restored would be beneficial. Also doing a HEAD request against a script to see if it has changed within greasemonkey (and recommend updating) might be possible. S3 means flexibility - at a cost - and right now that cost is cheaper than buying more hardware as the site grows.

BTW, it took longer to write this post than it did to do the conversion.

My Projects

• Book Burro
• Userscripts.org
• Taboo
• S3://
• WiiMe

more of my projects →