CTO/VP Engineering – What’s the Difference?

Mark Suster has an excellent post entitled “Want to Know the Difference Between a CTO and a VP Engineering?”

In his post, Mark discussed both the attributes of a CTO and VPE and when they are needed in a company’s lifecycle. In summary:

  • CTO – visionary, technically astute, not a great people manager
  • VPE – technical (knows how to code), process-oriented (builds, unit tests, automation, schedules) and can manage people

He also has some interesting thoughts from a VC viewpoint regarding about teams that have a consulting firm build their initial product

If you want to build a great technology company, you’ll need a “rockstar” engineering lead.  Every great tech startup needs one.  Whenever I meet a team that had a consulting firm (even a great one) build their product it’s an immediate “pass” from me.  If you don’t have somebody

inside your organization who is setting the technology direction then I’m convinced you’ll never head for greatness.  I know this will fall like a lead balloon to the many people who believe it is possible to have a [insert: startup incubator or technology accelerator or technology consultant or outsource firm] build your technology.  I don’t believe it.  Either your core is innately technical or it’s not.  It’s what makes Google Google and Facebook Facebook.

Mark suggests the proper time to bring in a VPE is when the CTO is managing more than 3 developers.

Based on my experience, I generally agree with Mark, however, I would add a few things:

  1. The VPE *has* to write code when the team is small (less than 10 people). I have been an advisor to two companies where the VPE managed a team of 4-6 people and did not code. They generally were clueless about the architecture, the process of actually getting things done (e.g. where the SVN drop was, how to build, write a unit test, deployment, etc). At a company of this size, the VPE should be a contributing member of the team; in both situations the VPE was ineffective and ended up leaving the company.
  2. Adding a VPE and CTO when the company has 4 or more developers seems really (!) top heavy. Personally, I would expect that the CTO could scale a bit better than that, even if the CEO has to help out with some of the “softer” skills on the people management side.

Amazon Ships AWS SDK for Java

Amazon has shipped an AWS SDK for Java (announcement)

Immediate areas of interest for me are:

  • EC2
  • Elastic Load Balancing
  • S3
  • SQS

Now if they would just ship a version of the Product Advertising API (which lets you search Amazon’s ginormous product DB) for Java – it’s been broken since August 15, 2009 when Amazon started requiring signed headers:

Dear Developer,
We want to remind you that all calls to the Product Advertising API must be authenticated using request signatures by August 15, 2009. Please remember that calls to the Product Advertising API that are not signed will not be processed after August 15, 2009. For help on request signatures, please see the Resources section below.
The Product Advertising API Team.

Their Java library does not support this functionality and has not been updated, leaving Developers to follow long posts like this to try and use the Product API. Seems like Jeff Barr or another Amazon Evangelist would want to fix this problem. I’ve only been asking since November 2009 …

Quora – A better FriendFeed/Aardvark?

Dave was kind enough to send me a Quora invite last week. After a few days of trying out the service, I am already hooked.

Quora has a lot of the elements of two services I love: Friendfeed and Aardvark. As Friendfeed is working towards a slow death, I’ve been on the lookout for a something to possibly replace it (I realize that  Quora does not centralize all my tweets etc etc but it *does* provide a place to engage in dialogue about topics of interest, which was always the true value of Friendfeed.)

I’ve made a n00b mistake or two, confused by some of the UI (like this):

Now I’m hoping they expose an API so I can create a killer Firefox add-on like this (which I just submitted an update for 3.6 and up…waiting for approval)  Maybe I can talk sandosh into helping …

Queuing Tweets with Amazon’s (AWS) SQS

Like many services, we wanted to enable social sharing on Twitter (and other platforms like Facebook) at the launch of PersistentFan.

Initially, our interface with Twitter worked reliably and allowed people to tweet about interesting videos. A basic flow diagram looked like this:

However, once we opened up PersistentFan, the increase in traffic (which wasn’t a tidal wave) we noticed a lot of tweets weren’t showing up on Twitter.

Our first attempt was to simply add a basic retry mechanism, which improved things, but only slightly. Our second attempt was to retry multiple times. The flow diagram turned into something like this:

 

Unfortunately, several times this led to site performance issues as server threads were busy trying to deliver a tweet instead of handling an inbound HTTP request

I had solved a similar problem (at a much larger scale) at MessageCast by adding message queuing. Implementing a full blown system with ActiveMQ etc seemed like overkill so I took a look at AWS SQS. It was really easy (and quick) to build a basic prototype. After looking at the typica library, I ended up choosing the AWS library and was off and running.

Tweet delivery logic has now changed to:

 

Overall, things were up and running pretty quickly and we’ve had very few issues. Currently, the AWS console doesn’t allow you to peek into a given queue and manage it (i.e. modify, delete etc) but this is supposed to be added in the near future.

Here’s some basic code for sending and getting to/from a queue:

Sending a message to the queue:

SendMessageRequest request = new SendMessageRequest();
request.setQueueUrl(<QueueURL>);
request.setMessageBody("This is my message text.");
invokeSendMessage(service, request);

Getting a message from the queue:

ReceiveMessageRequest request = new ReceiveMessageRequest();
request.setQueueUrl<QueueURL>;
invokeReceiveMessage(service, request);

Pricing is quite reasonable as well – 10k requests is $0.01

If you’re looking for a queuing solution, I’d recommend taking SQS for a spin.

Iterating in the Open

Over the years I’ve created, been an Advisor to and invested in online services. There have been various launch strategies from “let’s release as soon as we have these ‘n’ features”, “let’s release when we have feature parity with competitor ‘x’” and “hell, let’s release what we have and iterate as we go”.

There are pros and cons to each of the above three strategies. Based on my experience, the first and third ideas are workable, but waiting for feature parity is equal to never actually shipping anything (I’ve seen this several times).

When Mike and I came up with the idea for what is now PersistentFan (starting as an FB app, “top3Clicks”) we were determined to employ strategy number three, iterating in the open. The vision behind PersistentFan was to create a fan-oriented site where we could create a system that would programmatically acquire content about our various niche (or even micro-niche … is that an actual term??) areas of interest, notify us and enable sharing with our friends. We started off with something easy – video. Obviously, there are other types of content for a given area (news, blogs, photos, audio, etc) but YouTube had great APIs to get the ball rolling.

We aimed to iterate several times a week, pushing new features and bug fixes (here’s a sample). As the weeks flew by, the functionality of the site would increase bit by bit until we had a full featured offering. (Note that we did start off in bare-bones, early alpha invite-only mode). Feedback from friends (initially) and as the site grew, external users would help guide both the features and the priority of the features we shipped. Not crowd sourcing as described in Don Tapscott’s “Wikinomics” but instead open iteration, warts and all.

It’s been several months since we had the first user take the site for a spin and so far, here’s what we’ve found:

The good parts:

  • The ability to evolve service based on reality (i.e. user feedback, actual utilization)
  • Feedback based on usage instead of looking at a PowerPoint slide
  • Users pushing up the priority of a feature (we’ve had the request for a “Forgot My Password” several times now *cough*)

The not so good parts:

  • A new visitor to the site may not see value on initial visit and never return (the “is that all there is?” problem)
  • Bugs, bugs, bugs. We have a staging environment and plenty of automated tests, but when you’re running fast …

Stuff we’ve learned along the way:

  • Iterating in the open is a net positive, but users need information  describing updates and bug fixes (blog posts are great for this)
  • Don’t forget to mail registered users about updates (*cough*)
  • Enable the site with an open feedback mechanism (e.g. uservoice)
  • Have a strong grasp on analytics to see actual utilization (e.g. number of signups)
  • Have internal metrics as well as external (use Google Analytics) to get a good picture of what users are doing and the conversations about your site.
  • Reaffirmed that cloud computing is a fantastic way to scale super cheaply (AWS/EC2)
  • Automate as much as you can including unit tests, build and deployment scripts, etc. The time savings and reduction in errors pay off quickly.

Become a PersistentFan if you haven’t already, let us know what works/what doesn’t and look for a steady stream of changes. (Any questions why I’m the Mayor at the local Starbucks?)

Random Collection of Useful Tools

The effort on PersistentFan has led me to seek out various development/build/performance tools lately.

Some of the ones I found particularly helpful are:

  • Minify/YUI Compressor – minimizing the number of HTTP GET requests for CSS and Javascript resources and the size (in bytes) of these resources can increase client-side (browser) perf. Matt Snider has a great post on how both to concatenate and compress resources via Ant.
  • JSLint: Great service that analyzes your Javascript for code quality. Even better, RockstarApps has encapsulated JSLint to run as an Eclipse plug-in. Haven’t attempted to get this to run as part of the build (with Ant) yet.
  • Page Speed – I tried YSlow (add-on to Firebug) but found Google’s tool to be better overall. Really helpful in determining the best way to increase performance.

PersistentFan Graduates to Beta

Bacon has a post up about PersistentFan leaving invite-only mode.

We’re not quite ready for primetime, calling it an early Beta instead. Like many things, we have a zillion ideas to try out and will continue to rev the service. Check it out if you have a chance.

Here’s a few sample channels to test drive with:

You can also see what other people are watching

If you are a big fan of Jersey Shore or want to keep up with the latest from the disaster in Haiti, PersistentFan enables you to:

  • Create channels of interest that persistently search for new content
  • Receive email notifications when new content is available (you receive these as they are discovered or on a daily or weekly basis)
  • Share your favorites to Facebook and/or Twitter
  • Keep up with video feeds of Twitter’s trending topics

PersistentFan was built with open source, Java, Linux, Apache, Tomcat, mod_jk, mysql, junit, Struts, AWS (EC2 and S3), Javascript and plenty of Last.fm.