AWS Rolls Out Beanstalk

Amazon is continuing on their crazed march, adding yet another impressive set of features to AWS.

Today they announced the Beta launch of “Beanstalk”, which significantly reduces the complexity of service deployment and management. Previously, an AWS user had to deal with all the capacity issues:  provisioning, deployment/rollback, instantiation, load balancing and service monitoring. (Note that there are commercial solutions to this like RightScale and open source projects like OpenStack)

The initial offering is tailored to Java developers and uses Tomcat as the app server. All that is required is a .war file uploaded to Beanstalk that you then deploy. Beanstalk handles load balancing of the service and includes monitoring. Based on the post, it looks like there is some ability to control the environment (JVM settings etc) and directly login to a provisioned instance.

Of course, for the auto-scaling functionality to work properly, a site needs to be architected to work in an ‘n’ app server framework. I’d suggest testing this out before turning over the keys to Beanstalk.

Pricing for Beanstalk is attractive (free). The basic deployment looks incredibly inexpensive as well.

AWS Elastic Beanstalk Pricing

SEO and Sitemaps on

Sitemap Bacon and I are always working on search engine optimization (SEO) for In addition to ad buys, viral features, etc., like many sites, we have a monthly spend for acquiring traffic. Also like many sites, we constantly tweak many aspects of this spend to optimize.

Over the last few months, we added features to improve the visibility of a given topic (like Justin Bieber) by tweeting new topics from our @PersistentFan account, making it easier to share a link for a specific video (e.g. Jimmy Fallon), and Facebook/Digg sharing. Search engine crawlers are fairly latent; it can take upwards of a month to see the impact of certain changes. As we introduced various features, we began to see an uptake in both eCPM and search engine traffic/visibility. However, the progress was fairly muted overall. Before the holidays, we decided to start back at the beginning and analyze how search engines were crawling our site. I spent a lot of time doing this:

tail -f /var/log/httpd/access_log

One thing that struck me almost immediately was that I saw a ton of crawlers (Tweetmeme, etc) from the @PersistentFan tweets, but didn't see a lot of folks like GoogleBot. After some analysis, we noticed the obvious - the front door of showed both the most popular topics of the last few hours, along with a scrolling list of recently viewed videos. At any given time, we were exposing about 15 or 20 topics to bots, but no more. The many thousands of topics we track for persistent fans were effectively hidden from bots because we didn't provide direct linkage or navigation to discover them. (doh!)

The solution to this was to provide a sitemap. Since our topics grow dynamically, both from user-generated searches and topic creation and various feeds that we use on PersistentFan, we needed a solution that generated a sitemap on a regular basis. I looked around but couldn't find much in the way of libraries so in the end I wrote a Java app that grabs all of the topics in our database, builds and serializes a DOM to create a sitemap on a nightly basis. Once this was complete, a simple change to our robots.txt let the crawlers of the world know where to go:


Within a few hours we began to see crawlers performing GETs on URLs which were not previously requested. Sitemaps, FTW!

RSS is Dying (Again)

Kroc Camen has a post about the impending death of RSS.

RSS Button Having been a strong advocate of RSS (and also ATOM) for years (see MessageCast), I tend to read posts/rants about the soon-to-be-dead data format with some suspicion. Some posts are pure link bait (who could forget this?) While some are much more reasoned.

Kroc Carmen’s post falls into the latter category – he obviously values RSS and in his view, the lack of adoption (Mozilla claims a measly 3 - 7% utilization of the RSS “subscribe” features and is removing them from Firefox 4.0) is due simply to poor Web UI

The browser RSS button is the worst piece of UI since 2004.

I hadn’t stopped to consider this. After Firefox and IE added RSS support 5+ years ago things seemed to moving in the right direction. However, it was obvious that RSS is a geek-type thing that was never going to be adopted by the masses. Most people just don’t get what structured data feeds are all about (and of course, calling RSS “structured” is being kind)

Asa Dotzler (who btw follows 0 folks on Twitter??) has some great points in the post as well, check it out.

Eclipse Day at the GooglePlex

Last Thursday (8/26/2010) I attended Eclipse Day, held once again at Google.

In addition to the keynote by Ian Skerrett from the Eclipse Foundation, there were twelve presentations and an “unconference” session with 8 different topics of interest.

Overall, Eclipse Day rocked! Great presenters, some extremely relevant to my world including:

There were also presentations on mobile, git and various toolsets. (All the presentations are here)

If you use Eclipse, I recommend checking this out next time they come to Mt View

Iterating in the Open Presentation

I’ve been working with the folks at SSE Labs (aka the “Stanford Incubator”) this summer and as a Mentor, I was asked to give a presentation to the current teams.

Based on previous experiences and the work Bacon (Mike) and I did with PersistentFan, my presentation focused heavily on how to launch with a minimum desirable product and iterate quickly. As you might guess, it emphasized openness instead of going stealth.

Presentation is embedded below:

Lunch with the Interns

I was recently invited to attend a luncheon for new Microsoft interns.  They wanted me to discuss my background and thoughts about how to be successful in a tech-focused career.

After thinking about it for a while, I came up with three points to share with the group – I thought it was interesting enough (based on the questions/feedback) to share here:

  1. Be passionate about what you do – the best work experiences in my life have been when I’ve worked on things I am extremely passionate about. Conversely, the least favorite experiences have been when I’ve worked about things where my interest level is lukewarm.

    Life is short; spend your time working on things for which you have a true passion. These will vary by person but have similar characteristics. You’ll know you’re in deep when you don’t notice/care what time/day it is. When your bills start piling up because you’re never home. When your significant other starts calling herself a “startup widow”. When you’ve lived in the same place for three years and you meet some neighbors who tell you that they thought your place was “vacant for the last three years”.

    If you find yourself in a position that you aren’t passionate about, move on and do it quickly.

  2. Stay up to date – Mary Meeker states (Slide 5) that there will be more Smart phones shipping in 2012 than desktops/laptops. There wasn’t an intern in the room whose undergrad curriculum focused on this – the point being that things are forever changing in technology. Whatever your area of passion is (from #1 above), once you embark on a career you have to constantly work to stay abreast of the changes via blogs, thought leaders on Twitter, etc.

  3. Get your hands dirty – experience increases your value immensely. Whether this means choosing from a wider variety of projects within a company and/or ability to raise venture and start a company. The best way to gain experience is to dive headfirst into a project and immerse yourself. If you’re a developer, the best thing (IMO) to do is to write code as much as possible. Sitting in meetings all day won’t get you to the magic 10,000 hours you are going to need to become an expert.

Customer Loyalty and Rewards with Location-based Services

I’m the Mayor of my local Starbucks (aka the Peninsula office for PersistentFan) and regularly check-in on foursquare to retain my title.

This morning, I placed my usual order and was informed that my drink would be gratis as a reward for being the Mayor.

Very cool!

(A while back I checked into my fifth different Starbucks and received the Barista badge but wasn’t offered any discounts or coupons.)


There have been a ton of posts about Foursquare, GoWalla etc working with brick and mortar retailers to implement reward systems. The opportunities to utilize location data are large and some of scenarios are pretty compelling (witness the rumored $100m+ acquisition of foursquare )

Having now experienced one of the aforementioned customer rewards scenarios, I am even more bullish (and more of a fanboi) than I was yesterday. Now if I could just win back the Mayor of Taqueria La Bamba from Norm Y …

CTO/VP Engineering – What’s the Difference?

Mark Suster has an excellent post entitled “Want to Know the Difference Between a CTO and a VP Engineering?”

In his post, Mark discussed both the attributes of a CTO and VPE and when they are needed in a company’s lifecycle. In summary:

  • CTO – visionary, technically astute, not a great people manager
  • VPE – technical (knows how to code), process-oriented (builds, unit tests, automation, schedules) and can manage people

He also has some interesting thoughts from a VC viewpoint regarding about teams that have a consulting firm build their initial product

If you want to build a great technology company, you’ll need a “rockstar” engineering lead.  Every great tech startup needs one.  Whenever I meet a team that had a consulting firm (even a great one) build their product it’s an immediate “pass” from me.  If you don’t have somebody

inside your organization who is setting the technology direction then I’m convinced you’ll never head for greatness.  I know this will fall like a lead balloon to the many people who believe it is possible to have a [insert: startup incubator or technology accelerator or technology consultant or outsource firm] build your technology.  I don’t believe it.  Either your core is innately technical or it’s not.  It’s what makes Google Google and Facebook Facebook.

Mark suggests the proper time to bring in a VPE is when the CTO is managing more than 3 developers.

Based on my experience, I generally agree with Mark, however, I would add a few things:

  1. The VPE *has* to write code when the team is small (less than 10 people). I have been an advisor to two companies where the VPE managed a team of 4-6 people and did not code. They generally were clueless about the architecture, the process of actually getting things done (e.g. where the SVN drop was, how to build, write a unit test, deployment, etc). At a company of this size, the VPE should be a contributing member of the team; in both situations the VPE was ineffective and ended up leaving the company.
  2. Adding a VPE and CTO when the company has 4 or more developers seems really (!) top heavy. Personally, I would expect that the CTO could scale a bit better than that, even if the CEO has to help out with some of the “softer” skills on the people management side.

Amazon Ships AWS SDK for Java

Amazon has shipped an AWS SDK for Java (announcement)

Immediate areas of interest for me are:

  • EC2
  • Elastic Load Balancing
  • S3
  • SQS

Now if they would just ship a version of the Product Advertising API (which lets you search Amazon’s ginormous product DB) for Java – it’s been broken since August 15, 2009 when Amazon started requiring signed headers:

Dear Developer,
We want to remind you that all calls to the Product Advertising API must be authenticated using request signatures by August 15, 2009. Please remember that calls to the Product Advertising API that are not signed will not be processed after August 15, 2009. For help on request signatures, please see the Resources section below.
The Product Advertising API Team.

Their Java library does not support this functionality and has not been updated, leaving Developers to follow long posts like this to try and use the Product API. Seems like Jeff Barr or another Amazon Evangelist would want to fix this problem. I’ve only been asking since November 2009 …

Quora – A better FriendFeed/Aardvark?

Dave was kind enough to send me a Quora invite last week. After a few days of trying out the service, I am already hooked.

Quora has a lot of the elements of two services I love: Friendfeed and Aardvark. As Friendfeed is working towards a slow death, I’ve been on the lookout for a something to possibly replace it (I realize that  Quora does not centralize all my tweets etc etc but it *does* provide a place to engage in dialogue about topics of interest, which was always the true value of Friendfeed.)

I’ve made a n00b mistake or two, confused by some of the UI (like this):

Now I’m hoping they expose an API so I can create a killer Firefox add-on like this (which I just submitted an update for 3.6 and up…waiting for approval)  Maybe I can talk sandosh into helping …