Hybrid Clouds - Silicon Valley Cloud Computing Group

Last night I attended the Cloud Computing Group’s meeting on Hybrid Clouds.

The guest presenter was Paul Lancaster from GoGrid. Paul attempted to get through his slide deck (about 12 slides) in a timely fashion. However, there were so many questions/opinions that his presentation took over an hour. The questions varied widely – one high-level observation is that “cloud computing” still means many things to many people. Also, the cloud experience level of the audience was all over the map, with some just learning about the concept and others quite experienced with a particular vendor (AWS, etc).

GoGrid’s particular cloud offering was interesting and quite a bit different than AWS. However, being the AWS fanboy that I am, I didn’t find GoGrid compelling enough to warrant a switch. Of particular note, they are privately funded with ~130 employees. Given the costs of infrastructure, I would worry about their ability to geolocate, QoS and scaling in general.

I’m looking for a copy of the slidedeck and will add if I can find it online.

Top 20 VC Bloggers OPML

Eric Schonfeld has a Techcrunch post this morning on the latest edition of Larry Cheng’s Top 100 VC bloggers (list is based on the number of subscribers each blog has on Google Reader)

To avoid having to add each blog manually in your feed reader, I made an OPML file of the Top 20 bloggers on the list available here



Quick refresher on how to import OPML into Google Reader from the FAQ:

11. Can I import an existing subscription list?
If you're switching from another feed reader, you can import your existing subscriptions into Google Reader. To do this, you first have to export your subscriptions in a standard format called OPML (see the next question to learn how to do this). Once you have your OPML file, go back to Google Reader and click "settings" at the top right, then "import/export." Choose the location of your OPML file and click "import"; Google Reader will add all the subscriptions in the list to your account and start checking them for updates

What is “Cloud Computing” and What is the Future Valuation?



How *do* you define “cloud computing? Interesting article that examines how different research organizations are defining the term.

Gartner says:

a style of computing where scalable and elastic IT capabilities are provided as a service to multiple customers using Internet technologies

While a UC Berkeley paper offers:

Cloud Computing refers to both the applications delivered as services over the Internet and the hardware and systems software in the data centers that provide those services. The services themselves have long been referred to as Software as a Service (SaaS), so we use that term. The data-center hardware and software is what we will call a Cloud. When a Cloud is made available in a pay-as-you-go manner to the public, we call it a Public Cloud; the service being sold is Utility Computing

People in the industry don’t necessarily agree with such a broad view of the cloud:

If I define the cloud the way Gartner does, I could conceivably consider any Internet-delivered service as a cloud service," Treadway said. "That's not a helpful definition from the standpoint of the massive shift that's going to happen over the next 10 years in computing architecture. Gartner is diluting the term and making its figures irrelevant. Other experts don't defend Gartner's definition. Gartner is at odds with the industry

Gartner forecasts the cloud computing valuation at:

  • $46b in 2008
  • $150b in 2013

While IDC says the cloud will be worth:

  • $42b in 2012

Merrill Lynch says it will be:

  • $160b by 2011

My take: really hard to say (how’s that for helpful?)

As an entrepreneur, “cloud computing” really equates to utility/on-demand computing. The ability to provision virtual instances on the fly and scale as demand/traffic requires. Throwing out the old physical data center model and all the fixed costs that go with it, including hardware depreciation.

As a user though, it is really about the applications I use on a regular basis that exist in “the cloud”. Things like gDocs, Mint, Twitter, Facebook (and of course, Backpack).

In the end I suspect that the larger view will be used, if only because it makes the overall (revenue) numbers so much larger. Like anything, the potential will be overvalued in the near-term and undervalued in the long-term. Kinda like the web was back in the Web 1.0 days.

Now, who is going to acquire Amazon (AMZN) for their AWS technology??

Legalities of the Cloud

I finally finished “Cloud Application Architectures” (the Tour de France has been a real distraction of late – go Lance!), which is a great overview of cloud computing in general and utilizing AWS specifically.

One issue the author George Reese raises in the book concerns potential legal issues/concerns when your bits are cloud-based. For example, your virtual host is running on the same physical machine as another company. That company turns out to be under investigation for some shady dealings. Law enforcement officials in turn confiscate the physical hardware to prosecute the offenders, thereby taking your site down, along with your bits, including customer data. There are a number of other examples in the book, along with suggested ways to keep your data safe (encrypted file systems, etc)

When reading the book, I thought some of these ideas might be a bit outlandish. Until I read an article on CNet today, entitled “Lawyers shine light on real cloud concerns”. James Urquhart covers some of the same ground as George Reese, while adding in some additional topics/questions like this post from an employment law attorney:

From an employment law perspective, I have not seen much, if anything on the subject. For example, Connecticut's wage and hour laws require employers to keep track of various records of the employee including hours worked, etc. The catch? Such records need to be kept at the employer's place of business for three years. Does storing the information in "the cloud" satisfy that?

Good food for thought; obviously the legal system won’t catch up to the realities of the cloud for quite some time so it pays to delve into something you might not look into normally when deploying a physical production instance at a data center.

AWS: Pushing Dev/Test Environments in the Cloud

The AWS Start-up Tour, combined with a few books I’ve been reading lately (like the O’Reilly title “Cloud Application Architectures”) have made me re-think/see additional uses for cloud computing. Specifically, there can be a significant cost-savings when moving Dev/Test environments into the cloud.

(I’ll talk about the advantages of being able to quickly build/tear-down environments using images (AMIs) and EBS snapshots of a database in a different post.)

Using a hypothetical example, let’s say your startup/team has several test environments: Test, Stress, Stage and Production. The architecture is cloud-friendly enough to be able to think about moving at least your Test environment into the cloud. The question is, should you?

Hardware Requirements/Costs

The physical machine layout for Test might consist of the following:

Server type Cost Qty Total
Web  $3,000 1 $3,000
Application $4,000 2 $8,000
Database $4,000 2 $8,000
Cache $4,000 2 $8,000
    Total $27,000

(I’m excluding costs like bandwidth, rack space, hardware depreciation etc for simplicity. Including them only makes the case more apparent)


The initial costs come to $27,000. Note that it is definitely possible to incorporate cheaper hardware (and these are only estimates, YMMV). In my current environment, these costs would be significantly on the low-end.

Release Cycle

Following a basic “agile”/deliver-often methodology, the project plan (you do have one, right?) might roughly look something like this:

Week(s) Purpose
1 Planning
2-5 Coding
6-7 Test
8 Deployment

In the above example, Test consumes approximately 25% of the release cycle. (If the cycle time goes to 4 weeks instead of eight, the ratio stays the same). In other words, 75% of the time, the hardware is not utilized.

Cloud Hardware Requirements/Costs

Let’s assume that the Test environment is used 20 hours/day, 5 days/week during the Test phase of the release cycle (20 hours x 10 days or 200 hours). Note that we only pay when we have instances running. We can tear them down and not pay for the overnight/weekend hours. If we build the same infrastructure in AWS, we might see something roughly like this:

Server type Cost/hour Qty Total
Web $0.10 1 200 x $0.10 = $20.00
Application $0.20 2 2 x 200 x $0.20 = $80.00
Database $0.40 2 2 x 200 x $0.40 = $160.00
Cache $0.20 2 2 x 200 x $0.20 = $80.00
    Total $360.00










Using AWS, the Test cycle would cost approximately $360 per release. There would be some initial ramp-up costs as the team learned the ins/outs of AWS.

If we focus solely on the hardware costs between cloud utilization vs physical utilization, we get a one-time cost of $27,000 (assuming no service/maintenance contracts, etc) vs a per-release cost of $360. Said another way, it would take 75 releases tested in the cloud to equal the cost of the initial hardware outlay.

I realize that some points of my hypothetical scenario are missing – however, I think this illustrates the significant cost savings that can be achieved by moving Test and Dev environments to the cloud. From my current vantage point, this is truly compelling.

Pretty much a no-brainer.

AWS Management Console: Support for CloudFront

Over on the Amazon Web Services Blog they’re announcing support for CloudFront in the AWS Management Console app. Great summary as well:

You can start distributing your content in minutes. You don't need to make a long term commitment and you don't need to download a client application. It is now even easier to access CloudFront in pay-as-you-go fashion.

I’m not trying to become an AWS fanboy, but these guys are on fire lately. Might be time to sell my Akamai (AKAM) shares.

Also, makes wonder if articles like this will end up being wrong in the end…

AWS Start-up Tour: Cloud Computing with Amazon

Yesterday (6/16/2009) I attended the Amazon “AWS Start-up Tour 2009” in Sunnyvale at the PlugandPlay Tech Center (which reminded me of a larger version of the The Enterprise Network (TEN) Incubator we were in when we started iPrint)

Update: Slides are here

The event centered around cloud computing and how to utilize the various components of AWS. We’ve been running top3Clicks on AWS (EC2/S3) for almost a year and a half so I was pretty familiar with the basic offerings.

Things were kicked-off by Andy Jassy who runs the AWS business (and authored the initial business plan). He gave a good overview of all that AWS has to offer, including their new offerings around scaling, CDN and monitoring.

The best part of the event were the case studies of several companies utilizing AWS for some or all of their production infrastructure:

  • Paco Nathan, Principal Scientist, ShareThis
  • Ljubomir Buturovic, Ph.D., Sr. Director and Chief Scientist, Pathwork
  • Santosh Rau, Engineering Manager, Software Infrastructure, Netflix
  • Andrew Gibbons, Director of Operations, Smugmug

ShareThis and Smugmug are running significant portions of their infrastructure on AWS and both are looking at how to increase their utilization of the platform. Pathwork and ShareThis were interesting in that both had batch scenarios where they needed to provision a large number of machines (more than 1000) for a period of time. Once the batch jobs were complete, they would tear down the instances, thereby reducing both their costs and need for hardware.

At least one company was also using AWS to fire up Dev/Test instances on-demand, run scenarios and then tear down the environment. Given that we’ve spent a lot of effort on something quite similar (using Hyper-V for virtualization of test instances) on my current team, I found it extremely interesting how quick/easy this could be accomplished. There wasn’t any detail, but one scenario might go something like:

  1. Create daily/nightly RC build
  2. Fire up EC2 instance(s) that replicate enough of the production environment to verify the RC
  3. Install/deploy nightly RC to cloud
  4. Run automated functional suite of tests
  5. Generate reports
  6. Tear down environment

No need to have a farm of boxes sitting around that are used for a few hours a day, taking up space in a lab or datacenter. Nice!

The final presentation was done by Jinesh Varia, Technical Evangelist for AWS. He walked through how to take an existing classic three-tier infrastructure and port it to AWS. He pushed the Presentation (html, images, etc) to S3, the Application layer to EC2 and the Data tier to SQS. It was a slightly simplistic view of the world (he only had 45 minutes) but left us able to extrapolate more complex cloud-based architectures.

Overall, a great overview of one company’s view of cloud computing. I left wondering again when Amazon (AMZN) will be acquired, not for their retail operation, but as an infrastructure play. Guessing it will be IBM and it will happen in the next year or so, especially given that Sun (SUNW) is going to Oracle (ORCL).

Two Twitter Apps of Value

Twitter is great but, as you might have noticed, missing a few features.

Two apps I’ve been using to help round out the Twitter feature set:

  • Twimailer – Twitter makes it tough to know when you have new followers. It is even tougher to know if you want to follow them back. Twimailer addresses this by pushing an email to you every time a person starts following you. They provide a fairly rich message, including the person’s profile, recent tweets and number of followers/following.
    A good number of follows I receive on a daily basis are pure spam. Twimailer makes it easy to quickly determine if the follow is legitimate and if I should follow the person back.
  • TweetBeep – Great tool to stay apprised of conversations you care about – people compare it to Google Alerts.
    Personally, I use it a bit differently; tired of missing @replies, I set up a search that emails me when people tweet replies to me (@davehod). Super useful, especially when I’m away from my Seesmic client for any length of time.

Ray Ozzie – Comments on Cloud Computing

An interesting quote from Ray Ozzie at the J.P. Morgan Tech conference today

“… at some point in time every major enterprise, every company, every ISV is going to have some blend of software that runs on-premises and some that runs in the cloud, and everyone wants tools that they can use to in essence deploy some apps to part of their organization that might be in the cloud …”

(h/t TechCrunchIT)

Nothing revolutionary is his quote, but it does re-emphasize my thoughts about Amazon/EC2 – when are they valued for their cloud computing technology? When are they acquired for their cloud computing technology?

Amazon Rocks The House with New EC2 Features

Amazon announced three new features for EC2 today:

  • CloudWatch – monitoring your instances
  • AutoScaling – automagically scale/shrink your instances
  • Elastic Load Balancing – balance traffic across instances, including geo-distributed clusters (nice!)

There are a lot of conversations going on about the new features. Looks like Amazon has significantly extended their lead in cloud computing. Who'd a thunk it?

Several things jump to mind for me:

* BaconMarathon – we've been running our top3Clicks Facebook app in the EC2 cloud for more than a year now. Overall, the experience has been quite good, but there were a few things missing, most notably a scaling technology solution. I looked at RightScale, but their price point was not for us. Additionally, basic monitoring (disk space, CPU, RAM, app-level like HTTP GET) would have saved us from a few outages and the heartburn that comes with it.

* AMZN p/e - when does Amazon's p/e ratio begin to reflect the technical leadership/lead that they have established in cloud computing? Or has this already occurred since $AMZN is trading at a current p/e of 48.64 while GOOG is 29.02? I saw just today that AT&T was launching a cloud initiative with EMC; great that they are coming to the party, but they are pretty late. Amazon is grouped in Retail - when does this change/does Amazon need to spin this out into a separate company to realize the true forward looking value of the technology? Or, when does an IBM acquire Amazon for the technology and sell-off the retail business? Even more far-fetched, when does Amazon start acquiring technology companies in an Oracle-type rollup strategy?

Great comment from Matthew Snape on my Friendfeed post:

Dave AWS is so brilliant because it is a retail operation. They sell CPU time in exactly the same way they sell books or dvd's, all you need is a credit card. Compare that with the way IBM or Oracle operate. Perhaps it will be Amazon buying Oracle?

* Startups - We've all seen this movie before, but when startups build functionality on top of another company's APIs, things usually go something like this:

  • Startup sees APIs, builds cool/missing functionality to fill in the gaps. Ignores comments that they are "just a feature"
  • Startup secretly hopes API publisher sees value and acquires startup
  • API publisher sees that startup has validated model and builds out same functionality
  • API publisher launches new features; startup ends up being "just a feature" and closes down















Perhaps folks like RightScale will figure out how to add more value/build out their cloud support to other offerings like IBM, Google, Microsoft, etc.