Using an m3.medium AWS server for less than the cost of a t1.micro

I've used Amazon Web Services for quite some time. My online football management simulator, MyFootballNow, runs on AWS.  The sims spin up on spot instances to help keep costs down, because they need to run on at least a m1.small server to finish in an acceptable amount of time. I also have a test environment where I try to sim a full season every day, to see the long-term effects of various code changes in the game engine. I've been using a physical server for that, because at this stage I don't want to pay for a server at Amazon full-time just to run a test environment. This server has been showing signs of impending failure lately, though, and rather than purchase a new server I began to explore the least expensive way to have a decent server at AWS.

My first thought of course was to purchase a reserved instance.  With the ability to sell your reservation if you end up not needing it any longer, the risks are low. But it's still a good amount of money up front for a system that isn't generating any revenue yet.

The absolute lowest cost for a server at AWS can be through spot instances.  When I changed running the sims from an on-demand server to a spot instance server, my costs reduced dramatically. Could I leverage a spot server to run my test environment?

If you're not familiar with spot instances, Amazon makes their unused compute capacity available at a discounted price.  You set a maximum price you're willing to spend, and if the spot price is below your bid then your instance launches, and you pay whatever the current spot price is.  If the spot price rises above your bid, your instance is terminated.  You can't stop a spot instance, you can only terminate it.  But you can set up a persistent spot request that will re-launch your spot instance from the AMI once the price goes back below your bid.

The biggest problem with the persistent request for me was that the instance essentially resets itself every time it launches.  This wouldn't be horrible, but it would mean making sure that I created a new AMI every time I made a significant change to the environment, and if I was shut down then I'd have to roll back to my most recent AMI.  There had to be a better way.

And there was.  When you create a spot instance (or any instance, for that matter) you are presented with the following screen as you create your volumes:

Note the checkbox on the far right "Delete on Termination" - if you uncheck this, the volume will stick around even after the instance has been terminated.  With this orphaned volume, we can recreate our instance at the point it was shut down.

The price of an m3.medium spot instance right now is a few hundredths of a cent less per hour than an on-demand t1.micro, and quite a bit less than a heavy utilization m3.medium instance.  The catch is that your instance could be shut down if there is a spike in the spot price, so take caution using this technique on a production server that you don't want to disappear unexpectedly.  For a development server, it works fabulously, and if you set your spot price bid high enough you might even be able to sustain your instance as long as you like.

So, what I did is create a spot instance of type m3.medium with a $0.02 max bid (the current rate is $0.008 and hasn't gone up even to $0.009 since Amazon's last price reduction).  That means I should pay no more than $15 per month for this server, and if the spot price never goes above my bid then I could potentially have my m3.medium for about $5/month.  You can of course extrapolate this to any server type.

Back to the catch: if your server shuts down, it will leave its volume behind, but if you're like me and have tried to launch a new instance from a volume in this state you probably failed miserably.  I finally figured out that the reason for this failure is because I was using the default kernel, which didn't match my previous server.  So, find the kernel id for your spot instance:

and make a note of it.  I have it in the name of my Volume, i.e. "Dev-01-/dev/sda-aki-919dcaf8" - which will give me everything I need to know to launch a new instance from this volume.

To launch an instance with your volume after the system has been terminated, right-click on the volume and create a snapshot.

Now go to the snapshot list, find your snapshot you just created, and create an image from it:

When you create the image, you are given the opportunity to choose the kernel ID - it is important to use the same kernel ID you discovered in the above step.  This is why I always name my spot instance volumes with the kernel ID.

My spot instance actually has two volumes plus the instance store - the instance store is gone forever when the instance is terminated, I just use it for scratch space. To apply the second volume, you'll need to create a snapshot from it and then choose the snapshot when you set up the volumes.  Make sure you have it mounted to the same location if you have an /etc/fstab entry for it; again, that's why my volume naming convention contains the /dev/sda part.

After you've created the image, make a new spot request with that image.  Once you have logged in to the new instance and verified that everything is as it should be, you can delete the old volumes, delete the AMI, and delete the snapshot.

You might also combine this with my script to set a DNS entry on boot, which will keep your server's DNS record up to date.  The only gotcha here is that the host identification will change with the new image; depending on how you are connecting via SSH it should not be difficult to reset the identification.

Hope this helps and enjoy your no commitment discount m3.medium servers!