May I Take Your Order? Our Menu of AWS Services

This is the second post in my series about how we use Amazon Web Services at Church on the Move to create highly available scaling websites. In my previous post, I talked about my experience in getting involved in Amazon Web Services, and how we ended up moving toward running our websites using servers in the cloud. Today I want to give an overview of the services we use at AWS to make our auto-scaling happen.

AWS has a smorgasbord of really cool systems. Several of the systems I know very little about. Most of the ones I know little about I have no real need to use - they offer services I don't need. I'm not running complex simulations or doing massive data conversions, or any of a number of other operations that would use those services. What I do is build and maintain websites and databases, usually integrated together, with fairly complex logic that includes user authentication, CMS, file storage, etc. We have a specific combination of AWS services that we use to accomplish this, and here I will give a brief overview of what those services are. I'll go into further detail on how we use each system - and hopefully have enough information that you will be able to build a similar system for yourself.

Elastic Cloud Computing

The backbone of what we use is the Elastic Compute Cloud, or EC2. For a while I thought maybe this was the second generation of their cloud servers, and then I realized that the 2 means there are 2 C's ... why do I mention that? Because even though I'm a programmer I do have an active random thought generator. I will get back on point now.

EC2 is Amazon's cloud computing system. The main part are the EC2 instances - virtual machines that you launch and pretty much act like a physical machine. Amazon offers several tiers of EC2 instances, ranging from a "micro" instance - which costs next to nothing and actually can perform quite well on a website that doesn't get a constant stream of traffic - to very large instances that provide lots of CPU power or memory or both, but are very expensive to run. AWS bills you by the hour that the instance is running. If you launch an instance for 2 hours, you only pay for those two hours. No long term commitment, no up front costs ... unless you purchase a "reserved" instance, which basically costs a relatively small amount up front and then you pay a significantly reduced hourly rate. Once you decide you will be using AWS long term, you  should purchase reserved instances to make your experience even more inexpensive.

One of the coolest aspects of using a virtual server instead of a physical one is the fact that you're actually working in an application with a file. In the same way that a Word document can be copied, saved, moved to another computer, and kept with multiple revisions, you can do those same things with your server. You can snapshot your server, which in essence is saving a copy of your drive. The corollary of that concept is creating an image of your server which you can then use to boot up an identical server. These concepts are awesome even if you aren't duplicating your server for auto scaling - I have servers that I don't have in any kind of auto scale group, but I take snapshots of them so that if something catastrophic happens - such as a rogue script deleting my entire hard drive, or a misguided "sudo rm -fr *" command, I can mount a drive from my snapshot and repair or rebuild my server.

Simple Storage Service

Next in line is Amazon's Simple Storage Service, or S3. If you don't know why it's S3, see my above random rant. I will stay on topic here.

S3 is basically an unlimited storage space that you can put whatever you want. (Well, as long as it's a file). Unlike an EC2 instance, where if you want to have a 1.5TB drive you have to pay for all 1.5TB each month, even if you are only using 3GB, S3 only charges you for the space you use and the time you are using it. You can make areas of S3 publicly available and even serve an entire static website from S3; you can make other areas of S3 highly secure and even use their server side encryption to remain HIPPAA compliant. I don't store anything where I have to worry about HIPPAA, so I won't go into any detail about that; suffice it to say that S3 is a key player in our website architecture.

Relational Database Service

RDS is Amazon's non-character-repeating managed database service. We use their MySQL servers, but they have other engines available as well. RDS puts the management of your database server into Amazon's hands. There are pros and cons to this, which I'll get into in another post. Most of our databases are running on RDS instances.

DynamoDB

Surprisingly not named DDB, this is Amazon's NoSQL database system. NoSQL is kind of a new buzzword in the industry, popular because of its speed and ease of use. I haven't jumped on the NoSQL bandwagon, because I am very much a relational database designer, but there are times that it makes more sense to use NoSQL if you don't need the relational features. I mention it here because we have some such usage. I'll talk more about how we use DynamoDB in a future post.

Amazon Cloud Watch

Clearly an indication that Amazon was growing tired of turning their services into Acronyms, Amazon Cloud Watch is their system monitoring service. The short description is that using Cloud Watch, you can be notified when certain metric thresholds have been hit, such as CPU usage of your servers or when your RDS instance has a certain number of connections. There are far more metrics you can monitor than I will ever care about. But, besides sending you e-mails and text messages when these alarms fire, you can also have them ping a webpage or execute auto scaling actions. We have different alarms doing all four of those actions, and obviously this is a pretty strong key when it comes to auto-scaling.

Route 53

Returning to cleverly named services, Route 53 is Amazon's DNS service. Unlike the DNS service you would use with your domain at DynaDot or with the name servers you got from your shared hosting service, you do have to pay for a Route 53 domain name. But there are a lot of reasons to use it, all of which I won't go into now. I'll just say that we use Route 53 because ... I prefer it. That should be good enough. And I suggest you check it out, you'll probably prefer it too.

That concludes the list of services I'll be discussing in this series. These are the services that I am most familiar with because I deal with them on an almost daily basis. And these are the services we use to build our auto-scaling system.

Briefly, I want to mention one other cool service called Elastic Beanstalk. This is Amazon's managed scaling solution. Basically everything I am going to cover in this series is done automatically if you use Elastic Beanstalk; however, I personally tried using this and didn't care for it. I prefer to have full control over my system's scaling activities and the servers that they run on. You may actually find Elastic Beanstalk fits your mold better; if it does, that's fantastic. It doesn't cost any more than an equivalent environment you built yourself.

Sunny Skies Ahead? My Foray into Cloud Computing

I recently was asked to give a presentation on how we use Amazon Web Services at Church on the Move to manage our websites, and especially how we use them to handle our Christmas Train sales - which have continued to expand in demand each year significantly. That got me thinking, I spent a lot of time going around the internet looking for how to put the various pieces together, but don't recall a single location where the information is easily available. This blog is my attempt at providing information that hopefully will help others who are interested in Amazon AWS and would like to see how someone is actually using their services in a website.

I would  like to point out that I do not consider myself to be an expert at AWS, but I have been using their services for a little over three years as of this writing, and have been around the block a few times. Hopefully my insights and experiences will help you make better decisions in regard to using Amazon AWS.

In this first entry, I am going to talk a little bit about cloud computing in general, my personal experience in getting involved in cloud computing, and an overview of the services we use at Amazon Web Services. I will go into more detail in future posts about each service and how it applies to our usage of Amazon Web Services.

As I mentioned above, I work at Church on the Move in Tulsa, OK as the Senior Web Applications Developer. That basically means that I am in charge of building systems to maintain our websites as well as building internal applications that are web-based. When I first started in this department, our website was hosted on a dedicated server in a data center somewhere - one of those black box data centers where their advertising shows beautiful rows of server racks, one of which is supposed to be yours, but in all reality is probably rows of inexpensive desktop machines sitting on shelves. Either way, we finally decided we had outgrown that and wanted to have our own physical server that we built and could actually see and touch. We found a local data center, and built a monster server to put there. This server (and all the servers we had on site) had 1.5TB of hardware RAID-5 storage - drives were cheap and we wanted to make sure we would never run out of room.

Then one day our church produced an awesome Father's Day tribute video called Dad Life. This video went viral, which drove a ton of traffic to our website, and the monster server we built ... didn't seem so monstrous any more. I managed to limp through that season and keep our site alive, but just barely. That was the first time in my professional career that we had a server overloaded with traffic.

Fast forward about 6 months, and we are gearing up for online sales for The Christmas Train - an annual event that our church puts on at our kid's camp called Dry Gulch, U.S.A.  It typically operates for 15 or so days between Thanksgiving and Christmas, and we welcome an average of 50,000 guests each season. The year prior was the first year that we offered online tickets, and was also the first year where our numbers were very underwhelming. This year was going to be different - we were going to offer a 24-hour super sale with heavily discounted tickets. Our goal was to sell half of the tickets during this sale.

After the Dad Life issue, I knew that we had the potential to be unable to withstand the potential demand given our current infrastructure. I began looking at AWS, because I had briefly been reading about how they have virtually unlimited compute capacity with auto scaling. The huge drawback was that it was fairly expensive to build a virtual server with 1.5TB of storage - at AWS or at any cloud computing service. Because this was how I was used to thinking, I had a difficult time deciding that AWS would be a long term solution for our web hosting. For Christmas Train, however, it was perfect. We ended up selling out that first year in about 35 hours. The second year we sold out in 11 hours. This past Christmas Train we sold out in 75 minutes - and would have probably sold out in less than 10 minutes if our infrastructure would have handled it.

Before I go on, I want to make a quick comment about that last sentence. A lot of people think that AWS and auto scaling is the magic bullet - just throw your system on an auto-scaling server and you'll never run out of capacity. I even had someone who had difficulty buying tickets this year say (in a not very nice way) that we could use something like AWS and we would never have any problems. But auto scaling is not a magic potion you apply to your website to prevent you from ever having any problems. You still have to anticipate accurately to some degree. Last year we sold out in 11 hours. This year we expected to sell out in 5-6 hours. We postured to sell out in 15 minutes. We would have sold out in less than 10 - probably less than 5 - if we could have. Doing that would have meant a fundamental change in our system design, because at that rate the issue was no longer server load, it was design decisions that were made with the expectation of a much slower sale rate. Accommodating the traffic we actually experienced would have meant completely changing the way the ticket sale process functioned - not just in the code design, but the whole end user experience as well.

We now have about a dozen websites served using on average 15-20 EC2 instances at Amazon AWS. My fears about the limitations of cloud computing proved to be unfounded, but I had to have a paradigm shift. You can't treat cloud computing in the same way that you treat a dedicated server. Sure, most of the ideas are the same, but the differences are not completely transparent.