June 29, 2009
AWS: Pushing Dev/Test Environments in the Cloud
The AWS Start-up Tour, combined with a few books I’ve been reading lately (like the O’Reilly title “Cloud Application Architectures”) have made me re-think/see additional uses for cloud computing. Specifically, there can be a significant cost-savings when moving Dev/Test environments into the cloud.
(I’ll talk about the advantages of being able to quickly build/tear-down environments using images (AMIs) and EBS snapshots of a database in a different post.)
Using a hypothetical example, let’s say your startup/team has several test environments: Test, Stress, Stage and Production. The architecture is cloud-friendly enough to be able to think about moving at least your Test environment into the cloud. The question is, should you?
Hardware Requirements/Costs
The physical machine layout for Test might consist of the following:
| Server type | Cost | Qty | Total |
| Web | $3,000 | 1 | $3,000 |
| Application | $4,000 | 2 | $8,000 |
| Database | $4,000 | 2 | $8,000 |
| Cache | $4,000 | 2 | $8,000 |
| Total | $27,000 |
(I’m excluding costs like bandwidth, rack space, hardware depreciation etc for simplicity. Including them only makes the case more apparent)
The initial costs come to $27,000. Note that it is definitely possible to incorporate cheaper hardware (and these are only estimates, YMMV). In my current environment, these costs would be significantly on the low-end.
Release Cycle
Following a basic “agile”/deliver-often methodology, the project plan (you do have one, right?) might roughly look something like this:
| Week(s) | Purpose |
| 1 | Planning |
| 2-5 | Coding |
| 6-7 | Test |
| 8 | Deployment |
In the above example, Test consumes approximately 25% of the release cycle. (If the cycle time goes to 4 weeks instead of eight, the ratio stays the same). In other words, 75% of the time, the hardware is not utilized.
Cloud Hardware Requirements/Costs
Let’s assume that the Test environment is used 20 hours/day, 5 days/week during the Test phase of the release cycle (20 hours x 10 days or 200 hours). Note that we only pay when we have instances running. We can tear them down and not pay for the overnight/weekend hours. If we build the same infrastructure in AWS, we might see something roughly like this:
| Server type | Cost/hour | Qty | Total |
| Web | $0.10 | 1 | 200 x $0.10 = $20.00 |
| Application | $0.20 | 2 | 2 x 200 x $0.20 = $80.00 |
| Database | $0.40 | 2 | 2 x 200 x $0.40 = $160.00 |
| Cache | $0.20 | 2 | 2 x 200 x $0.20 = $80.00 |
| Total | $360.00 |
Using AWS, the Test cycle would cost approximately $360 per release. There would be some initial ramp-up costs as the team learned the ins/outs of AWS.
If we focus solely on the hardware costs between cloud utilization vs physical utilization, we get a one-time cost of $27,000 (assuming no service/maintenance contracts, etc) vs a per-release cost of $360. Said another way, it would take 75 releases tested in the cloud to equal the cost of the initial hardware outlay.
I realize that some points of my hypothetical scenario are missing – however, I think this illustrates the significant cost savings that can be achieved by moving Test and Dev environments to the cloud. From my current vantage point, this is truly compelling.
Pretty much a no-brainer.
Posted by davehod at 05:20 PM | Permalink | Comments (0)
June 26, 2009
AWS Management Console: Support for CloudFront
Over on the Amazon Web Services Blog they’re announcing support for CloudFront in the AWS Management Console app. Great summary as well:
You can start distributing your content in minutes. You don't need to make a long term commitment and you don't need to download a client application. It is now even easier to access CloudFront in pay-as-you-go fashion.
I’m not trying to become an AWS fanboy, but these guys are on fire lately. Might be time to sell my Akamai (AKAM) shares.
Also, makes wonder if articles like this will end up being wrong in the end…
Posted by davehod at 08:47 PM | Permalink | Comments (1)
Maui: Kaanapali 10k Run
We spent the last week in Kaanapali and were able to get in some decent runs. The 10k below had some good hills (my Garmin says it was 1377 feet of incline), great scenery and was runnable at 5:30am HST.
Course directions
Starting out on Nohea Kai Drive:
- Head north on Kaanapali Parkway
- Turn right on Kekaa Drive, heading up the hill past the golf course
- At the top of the hill, take an immediate right onto Kualapa Loop (more uphill)
- Turn left at the 2nd stop sign (Puu Anoano Street). Look down the hill for a great view of the ocean and Lanai.
- Turn left on Puukoii Road and head the down the hill and cross Hwy 30 at the light.
- Turn left on Kai Ala Drive, run to the end of the court. Follow the “Beachfront This Way” signs (this is approximately Mile 4). This part is a little tricky, so stay on your toes.
- Follow Kekaa Drive to the bottom of the golf course, turning left on Kaanapali Parkway
- Just before the end of the street, look for the “Shoreline Access” sign/path. Follow this to the beach.
- Take the beach path until it ends. Cross the bridge, past the cemetery and continue on to Hanakaoo Beach Park (aka Canoe Beach). Take a quick breather as this is Mile 5.
- Turn around, run back along the path to Nohea Kai Drive.
Staying cool/hydrated is sometimes a challenge for me in humid environments (the Bay Area has very low humidity). I find my fluid intake is about 50% higher when running here vs. running at home. Also, cooling down takes longer; make sure to allow time to walk at least 5 minutes after completing the run.
Posted by davehod at 05:28 PM | Permalink | Comments (0)
June 17, 2009
10k Run on Catalina Island
Last weekend we ran a 10k on Catalina Island. Having never been there before, I headed over to the USATF website and found a straight-forward 10k.
Starting out in Avalon at the water’s edge, we ran north encountering a decent set of hills in a canyon (this was two weeks after the Newport Marathon, so my quads were quite vocal). At mile 2, we returned to town and headed west, past the (tiny) golf course and up the hill to the Botanical Gardens. Turning around at 4.5m, we again headed back to town and attempted to run out to Pebbly Beach. Unfortunately, due to a rock slide, the road was closed. We turned around, headed back to town and straight to Von’s for Gatorade.
After a quick cool down, we hit Jim’s for breakfast. Great day, I highly recommend it.
Posted by davehod at 12:14 PM | Permalink | Comments (0)
AWS Start-up Tour: Cloud Computing with Amazon
Yesterday (6/16/2009) I attended the Amazon “AWS Start-up Tour 2009” in Sunnyvale at the PlugandPlay Tech Center (which reminded me of a larger version of the The Enterprise Network (TEN) Incubator we were in when we started iPrint)
Update: Slides are here
The event centered around cloud computing and how to utilize the various components of AWS. We’ve been running top3Clicks on AWS (EC2/S3) for almost a year and a half so I was pretty familiar with the basic offerings.
Things were kicked-off by Andy Jassy who runs the AWS business (and authored the initial business plan). He gave a good overview of all that AWS has to offer, including their new offerings around scaling, CDN and monitoring.
The best part of the event were the case studies of several companies utilizing AWS for some or all of their production infrastructure:
- Paco Nathan, Principal Scientist, ShareThis
- Ljubomir Buturovic, Ph.D., Sr. Director and Chief Scientist, Pathwork
- Santosh Rau, Engineering Manager, Software Infrastructure, Netflix
- Andrew Gibbons, Director of Operations, Smugmug
ShareThis and Smugmug are running significant portions of their infrastructure on AWS and both are looking at how to increase their utilization of the platform. Pathwork and ShareThis were interesting in that both had batch scenarios where they needed to provision a large number of machines (more than 1000) for a period of time. Once the batch jobs were complete, they would tear down the instances, thereby reducing both their costs and need for hardware.
At least one company was also using AWS to fire up Dev/Test instances on-demand, run scenarios and then tear down the environment. Given that we’ve spent a lot of effort on something quite similar (using Hyper-V for virtualization of test instances) on my current team, I found it extremely interesting how quick/easy this could be accomplished. There wasn’t any detail, but one scenario might go something like:
- Create daily/nightly RC build
- Fire up EC2 instance(s) that replicate enough of the production environment to verify the RC
- Install/deploy nightly RC to cloud
- Run automated functional suite of tests
- Generate reports
- Tear down environment
No need to have a farm of boxes sitting around that are used for a few hours a day, taking up space in a lab or datacenter. Nice!
The final presentation was done by Jinesh Varia, Technical Evangelist for AWS. He walked through how to take an existing classic three-tier infrastructure and port it to AWS. He pushed the Presentation (html, images, etc) to S3, the Application layer to EC2 and the Data tier to SQS. It was a slightly simplistic view of the world (he only had 45 minutes) but left us able to extrapolate more complex cloud-based architectures.
Overall, a great overview of one company’s view of cloud computing. I left wondering again when Amazon (AMZN) will be acquired, not for their retail operation, but as an infrastructure play. Guessing it will be IBM and it will happen in the next year or so, especially given that Sun (SUNW) is going to Oracle (ORCL).
Posted by davehod at 11:33 AM | Permalink | Comments (0)
