Amazon just posted a security whitepaper which describes the security measures they have in place to protect their customers. It's a short read (9 pages) and I strongly recommend it if you are using EC2, S3 or SimpleDB for anything. They explain their security measures and make recommendations for further protection their customers can put in place to protect their data.

One major concern I had was addressed by the whitepaper, "The AWS proprietary disk virtualization layer automatically wipes every block of storage used by the customer, and guarantees that one customer’s data is never exposed to another." I was always curious about those disk devices on EC2 and what data might be lingering on them but never had the time to investigate.

Here's an interesting snippet regarding their physical security:
"Amazon has many years of experience in designing, constructing, and operating largescale data centers. This experience has been applied to the AWS platform and infrastructure. AWS data centers are housed in nondescript facilities, and critical facilities have extensive setback and military grade perimeter control berms as well as other natural boundary protection. Physical access is strictly controlled both at the perimeter and at building ingress points by professional security staff utilizing video surveillance, state of the art intrusion detection systems, and other electronic means. Authorized staff must pass two-factor authentication no fewer than three times to access data center floors. All visitors and contractors are required to present identification and are signed in and continually escorted by authorized staff."

At familylink.com, we have 4 MySQL database systems on EC2 that run that our facebook app, various other social network apps and various websites. I recently switched our disk storage for those instances from the standard EC2 instance disks to EBS (Amazon's persistent storage for EC2) and wanted to share some brief numbers with you regarding performance.

I'm using a simple (yet quite complex) metric to measure the performance increase, load. System load is a number that show how many processes are contending for system resources (usually CPU.) For a more detailed description of load, read this article.

Enough of the talk, here's what I saw when I switched the 4 databases over to EBS:
---Database server #1---
Purpose: 2 moderately used databases
Disk change: 2 striped local disks (raid0) to single EBS volume
Peak Load change: 2.5 to 1
Estimated disk performance increase: 5x

---Database server #2---
Purpose: 2 lightly used databases
Disk change: 2 striped local disks (raid0) to single EBS volume
Peak Load change: 1.5 to 0.5
Estimated disk performance increase: 6x

---Database server #3---
Purpose: 9 lightly used databases
Disk change: 2 striped local disks (raid0) to single EBS volume
Peak Load change: 1 to 1 (no noticeable change)
Estimated disk performance increase: 2x

---Database server #4---
Purpose: 1 heavily used database
Disk change: 4 striped local disks (raid0) to 4 striped EBS volumes (raid0)
Peak Load change: 3 to 1.5
Estimated disk performance increase: 2x

Keep in mind that in theory 2 striped disks are almost twice as fast as a single disk. That's why I say there's a disk performance increase of 2x on database server #3 even though there was no noticeable performance increase (we went from using 2 disks to 1 disk.)

There you go, real-world numbers from real-world sites and servers. In summary, it's safe to say you'll see a significant disk performance increase if you switch over to using EBS with your EC2 instances. In addition to the performance increase, it's a no-brainer that you want persistent storage for your databases. One other huge benefit is snapshots. You can quickly and easily snapshot your database for backup purposes or for testing/reporting you may want to run against your most recent production data. See Amazon's site for more details.

If you haven't yet tested EBS with your systems on EC2, now is the time.

Amazon just announced two new High-CPU instance types for EC2. Here's a quick comparison of what you can now get compared to the previous offering:
2x Small ($0.20/hr)
RAM - 3.4GB
CPU - 2 Compute Units*
DISK - 320GB
High-CPU Medium ($0.20/hr)
RAM - 1.7GB
CPU - 5 Compute Units*
DISK - 350GB

Extra Large ($0.80/hr)
RAM - 15GB
CPU - 8 Compute Units*
DISK - 1690GB
High-CPU Extra Large ($0.80/hr)
RAM - 7GB
CPU - 20 Compute Units*
DISK - 1690GB

So it really looks like a tradeoff between CPU and RAM. If you run more CPU intensive processes, then these new instance types may save you money.

*One EC2 Compute Unit (ECU) provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.

I recently had to alter(edit) a very large MySQL table on a production system to drop an unnecessary index. This particular table has over 15 million records in it and occupies over 6GB of space. Since this database runs the We're Related facebook app, I couldn't just bring it down for hours and make the change. This particular table stores data for a main part of the application.

Enter EC2. All of our production systems that power We're Related are running on Amazon's EC2 so this was an easy problem to work around. Using MySQL replication, I setup a system that would mirror this production database. The nice thing about replication is that it automatically keeps any mirrors up to date with the latest data.

Once I had the mirror up and running, I was able to run the command to alter the table on the mirror. This took over 6 hours to complete. Once done, it quickly caught up with those 6 hours worth of updates from the main system.

Tonight, I will be reconfiguring the webservers to connect to this new mirrored system instead of the original. This will result in less than five minutes of actual downtime.

End result:
downtime - < 5 minutes compared to 6+ hours
additional cost - $20 (for running an additional "beefy" server for 1 day)

That's one thing I just love about EC2, you pay for what you use. If you need a temporary server for hours or days, you just fire one up and pay minimally for it.

For us UNIX fans, Sun Microsystems is working with Amazon to offer OpenSolaris on EC2.
I'm not sure how I missed it, but they made the announcement last week.

I just signed up to be included in the beta. I'll post an update here when I get in and start tinkering. My main focus is going to be the performance of MySQL on EC2 comparing Linux with OpenSolaris.

I'm not sure what their long term plans are, but for now they aren't incurring any extra charges (beyond the standard EC2 charges.) I know that RedHat charges extra to run their Enterprise version on EC2.

Amazon just announced a price change for bandwidth for outbound traffic from EC2 and S3. The result is up to 26% reduction in costs for all outbound traffic depending on your volume. Here are the new prices which are effective May 1

New data transfer price (effective May 1, 2008)
$0.100 per GB - data transfer in
$0.170 per GB - first 10 TB / month data transfer out
$0.130 per GB - next 40 TB / month data transfer out
$0.110 per GB - next 100 TB / month data transfer out
$0.100 per GB - data transfer out / month over 150 TB

You can read full details here.

It looks like the easter bunny came a few days late for Amazon.

Yay! They finally offer static IP addresses for Amazon EC2. This is a huge step for anyone wanting to use EC2 to host anything on the internet. They are free of charge if you are using them with a running instance. If they are not associated with a running instance, there is a $0.01/hr charge for them.

This means that you won't have to worry about updating DNS with a new IP address if for some reason your instance shuts down.

They currently have a limit of 5 IP addresses per account, but you can request more if you need them.

They also have a new thing called "Availability Zones." This lets you put your instances (servers) in different locations allowing you some redundancy in case of some sort of failure of a location (like power failure or network problems.)

Facebook has announced a partnership with Amazon Web Services.

I don't quite see what the partnership involves other than some Facebook specific documentation and extra samples of facebook apps that run using AWS.

Either way, it's good to see some more promotion of AWS. As you know, I'm a big fan. We've been using AWS with our We're Related facebook app as well as the entire familylink.com website since October and love its instant scalability and cost savings. I'll be posting more about our Amazon EC2 experiences later.

I just released another Ubuntu public AMI for Amazon EC2. This one is nearly identical to the first one I released except it can be used with large and extra large instances.

Here are the current tech specs and costs for the different instance sizes:

Small Instance (Default) - $0.10/hr
1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of instance storage, 32-bit platform
Large Instance - $0.40/hr
7.5 GB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each), 850 GB of instance storage, 64-bit platform
Extra Large Instance - $0.80/hr
15 GB of memory, 8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each), 1690 GB of instance storage, 64-bit platform

I released my first public AMI for Amazon's EC2 service a few weeks ago. Last Monday, it got a 5 star review!

For those of you that don't know what EC2 stands for, it is Amazon's Elastic Compute Cloud. Still wondering what it is? You are not alone, it's not a very descriptive name.

Basically, you can start up your own server instance (or multiple instances) on Amazon's network for anything you want to use it for (legally.) An instance is like a server. The cost is between $0.10 and $0.80 an hour, depending on how powerful you want your instance to be. You can start up new instances in a matter of minutes.

For example, let's say my blog starts getting a ton of traffic because people finally realize how great it really is. As traffic grows, my little server probably won't be able to handle the load and people would start getting a "server is currently unavailable" message. To handle this sudden burst in popularity I could start up 20 servers on EC2 in less than 5 minutes, switch my site over there and people wouldn't get a "server is currently unavailable" message. Thus fulfilling the technological knowledge needs of millions of users every minute.

I had previously been doing some contract work for a few companies that were using EC2 and thought I should release the base server image I've been working on.

Here's a link to the page describing my Ubuntu 6.06 (Dapper) AMI