overstimulate

Amazon just released Elastic Compute Cloud (EC2). Amazon recently released scalable storage infrastructure (S3) and scalable (artificial) artificial intelligence (Mechanical Turk.) Now they have done the same thing for computation.

By creating (linux based) virtual machines, either from scratch or based on prebuilt ones (web, app and database servers), you can quickly deploy on a pretty powerful machine (1.7Ghz Xeon CPU, 1.75GB of RAM, 160GB of local disk, and 250Mb/s of network bandwidth.) You can remotely start and stop instances of your image using web service calls (as well as standard unix shutdown) and you are only charged for the time the server is running (since you have a dedicated machine.

You pay $0.10 an hour while the instance is running (and you can have multiple instances for more dimes.) Then you pay for bandwidth $0.20/GB. You can also send data to S3 for access to data when your EC2 is not running (normal storage fees for S3 apply, but you are not charged for the transfer between EC2 and S3.)

The possibilities this opens are large for the right kind of problems. For all those folks who are building YouTube clones, it should be much easier now! And at $876.00/year (not including bandwidth) it is very competitive with conventional hosting.

The product is in limited beta.

Getting Started

Step 0: have a linux box with java (or a mac)

Step 1: Create your account at http://aws.amazon.com

Step 2: Create a X.509 Certificate and download the private key and certificate to ~/.ec2

Step 3: Download the command line tools for working with EC2 (requres certificate from previous step)

Step 4: Setup environment varaibles (unzip and from within the folder it creates):

• export EC2_HOME=`pwd`
• export PATH=$PATH:$EC2_HOME/bin
• export EC2_PRIVATE_KEY=`ls ~/.ec2/pk-*.pem`
• export EC2_CERT=`ls ~/.ec2/cert-*.pem`

Step 5: List images: ec2-describe-images

This command should return a list available images. (on a mac you must set JAVA_HOME by doing: export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Home/ )

IMAGE   ami-68ae4b01    ec2-public-images/fedora-core4-base.manifest    206029621532    available       public
IMAGE   ami-69ae4b00    ec2-public-images/fedora-core4-apache-mysql.manifest    206029621532    available       public
IMAGE   ami-6bae4b02    ec2-public-images/getting-started.manifest      206029621532    available       public
IMAGE   ami-6dae4b04    ec2-public-images/fedora-core4-apache.manifest  206029621532    available       public
IMAGE   ami-6fae4b06    ec2-public-images/fedora-core4-mysql.manifest   206029621532    available       public

So we have Fedora Core with or without apache/mysql to choose from.

Step 6: Create a Keypair - ec2-add-keypair gsg-keypair

You will need to take the private key that it creates, save it to a file called id_rsa-gsg-keypair, then set it's permissions to be rw------- (chmod 600 id_rsa-gsg-keypair)

Step 7: Create an instance of your machine

ec2-run-instances ami-xxxxxxxx -k gsg-keypair (where ami-xxxxxxxxx is the id of the machine you used to use - found in step 5 with ec2-describe-images)

This step failed on me a few times, but eventually it reported that it created my instance.

RESERVATION     r-xxxxxxxx      ############    default
INSTANCE        i-xxxxxxxx      ami-xxxxxxxx            pending gsg-keypair

Step 8: Wait for it...

Using ec2-describe-instances you can check on the status of your machine, eventually it will display something like this:

RESERVATION     r-xxxxxxxx      ############    default
INSTANCE        i-xxxxxxxx      ami-xxxxxxxx    domU-##-##-##-##-##-##.usma1.compute.amazonaws.com      running gsg-keypair

Step 9: Authorize SSH & HTTP

ec2-authorize default -p 22 (ssh)
ec2-authorize default -p 80 (and http)

Step 10: Visit your page at the url given in the previous step

Now the fun begins!

You can ssh into the machine using the key you created in step 6
ssh -i id_rsa-gsg-keypair root@domU-##-##-##-##-##-##.usma1.compute.amazonaws.com and you will be greated with some ascii art

         __|  __|_  )
         _|  (     / 
        ___|\___|___|

 Welcome to an EC2 Public Image
                       :-)

Stats:

df

Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda1             10321208    780852   9016068   8% /
none                    870472         0    870472   0% /dev/shm
/dev/sda2            153915428    192676 145904308   1% /mnt

cat /proc/cpuinfo

processor       : 0
vendor_id       : AuthenticAMD
cpu family      : 15
model           : 37
model name      : AMD Opteron(tm) Processor 250
stepping        : 1
cpu MHz         : 2405.452
cache size      : 1024 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 1
wp              : yes
flags           : fpu tsc msr pae mce cx8 apic mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt lm 3dnowext 3dnow pni lahf_lm ts fid vid ttp
bogomips        : 4812.43

Lots more fun can be had, but I decided to try to setup a quick app

Yum included

I decided to test rails on the server, a quick setup based on Installing Ruby on Rails with Lighttpd and MySQL on Fedora Core 4. (install gcc, mongrel, and a few other friends)

You can check out the results here