Amazon Pricing Considerations

This page is a collection of notes and thinking out loud about Amazon Web Services (AWS) setups and their respective pricing considerations.

All prices assume US East (N. Virginia) region with a Linux system (Ubuntu)

Summary of Relevant AWS Considerations

There are a few important words/concepts to understand when deciding on what you want from an AWS setup.

EC2 (Elastic Cloud Compute): This refers to a virtual machine. When we talk about EC2 instances, we are referring to independent machines that you can ssh into. There are two kinds of storage mechanisms that an EC2 instance uses: EBS and instance store. By default, the root partition of an EC2 instance uses EBS. This is where the operating system is stored, and the /home/ubuntu directory, etc. The reason for this is that when the instance is shut down, the EBS persists. Instance storage, which is mounted (for example available as /mnt), can be large depending on the instance (e.g. m1.large has up to 2x420 GB), but this storage is lost on instance shutdown. Thus this should only be used for non-volatile storage.

EBS (Elastic Block Store): Permanent storage that must be attached to a machine to use. You can think of this as an external hard drive that Amazon plugs into the machine you tell it to. Amazon handles backing this up. You can also create snapshots to save old state.

S3 (Simple Storage Service): Web storage that does not need to be pre-allocated. The user defines a bucket that goes with their account and then you can upload as much stuff as you want, and access it whenever. This storage does not need to be attached to an EC2 instance.

ECU (Elastic Compute Unit): Roughly equivalent to a CPU.

Pricing Components

NOTE: We are ignoring I/O costs in the calculations below.

Source:

http://aws.amazon.com/ec2/pricing/
http://aws.amazon.com/ebs/pricing/
http://aws.amazon.com/s3/pricing/

EC2 Micro instance: $0.020 per hour
This comes out to $0.50 per day = $15 per month

EC2 m1.large instance: $0.240 per hour
This comes out to $5.76 per day = $173 per month

EBS: $0.05 per GB-month
For 1000 GB this would be $50/month or $600/year for storage.

S3: $0.085 per GB-month
For 1000 GB this would be $85/month or $1020/year for storage.

Case Study: Single Large Instance

Before going into the details of how to choose an instance and storage combo, let's go through a case study of a single large instance based on a single, large EBS instance. True, a simple demo setup of Millstone can be run under the AWS Free Tier, but any interesting project will require sufficient storage space to upload data and handle intermediate calculations. Thus the simplest implementation would be a single large instance that has all the computing and storage that you would need. For now, We won't consider local instance storage since it is lost when the machine shuts down. There might be a speed benefit to local instance storage, so that's an optimization TODO for later.

For a project like Fix rE. Coli which has 96 samples, we may need as much as 1 TB of storage, so we'll allocate an instance with that much EBS storage. We then want to create a machine with a few cores (one core per alignment) and at least 4 GB of RAM (this is what we've tested the pipeline with locally). Let's go with the c3.xlarge machine, which gives us 14 ECUs and 7.5 GB of RAM, costing $0.15/hour, or $108/month. This comes out to $158/month for instance and storage, assuming no optimizations.

However, since we only really need the CPUs while performing alignments, the idea is that we can then transplant the EBS to a smaller machine afterward.

Tangent: On EC2 Brain Transplants

Also see: http://alestic.com/2010/05/ec2-move-ebs-boot-instance

I thought I had come up with a simple solution to swapping the hardware running a monolithic instance of Millstone. The steps to the idea are:

1. Stop the instance you want to upgrade add detach its root EBS
2. Start a second instance with the desired hardware specs
3. Stop that second instance and detach its root EBS
4. Attach the root EBS of the first instance to this second instance [at root port /dev/sda1]
5. Start up the first instance (feel free to delete the second instance and its throwaway root EBS)

The problem with this idea is that it appears an EBS volume can only be bound to an instance in the same zone. If you can't get a new machine in another zone, then this doesn't work.

However, a potential solution to this problem is to clone the instance as an AMI and reassign it that way.