Iterating in the Open

Over the years I’ve created, been an Advisor to and invested in online services. There have been various launch strategies from “let’s release as soon as we have these ‘n’ features”, “let’s release when we have feature parity with competitor ‘x’” and “hell, let’s release what we have and iterate as we go”.

There are pros and cons to each of the above three strategies. Based on my experience, the first and third ideas are workable, but waiting for feature parity is equal to never actually shipping anything (I’ve seen this several times).

When Mike and I came up with the idea for what is now PersistentFan (starting as an FB app, “top3Clicks”) we were determined to employ strategy number three, iterating in the open. The vision behind PersistentFan was to create a fan-oriented site where we could create a system that would programmatically acquire content about our various niche (or even micro-niche … is that an actual term??) areas of interest, notify us and enable sharing with our friends. We started off with something easy – video. Obviously, there are other types of content for a given area (news, blogs, photos, audio, etc) but YouTube had great APIs to get the ball rolling.

We aimed to iterate several times a week, pushing new features and bug fixes (here’s a sample). As the weeks flew by, the functionality of the site would increase bit by bit until we had a full featured offering. (Note that we did start off in bare-bones, early alpha invite-only mode). Feedback from friends (initially) and as the site grew, external users would help guide both the features and the priority of the features we shipped. Not crowd sourcing as described in Don Tapscott’s “Wikinomics” but instead open iteration, warts and all.

It’s been several months since we had the first user take the site for a spin and so far, here’s what we’ve found:

The good parts:

  • The ability to evolve service based on reality (i.e. user feedback, actual utilization)
  • Feedback based on usage instead of looking at a PowerPoint slide
  • Users pushing up the priority of a feature (we’ve had the request for a “Forgot My Password” several times now *cough*)

The not so good parts:

  • A new visitor to the site may not see value on initial visit and never return (the “is that all there is?” problem)
  • Bugs, bugs, bugs. We have a staging environment and plenty of automated tests, but when you’re running fast …

Stuff we’ve learned along the way:

  • Iterating in the open is a net positive, but users need information  describing updates and bug fixes (blog posts are great for this)
  • Don’t forget to mail registered users about updates (*cough*)
  • Enable the site with an open feedback mechanism (e.g. uservoice)
  • Have a strong grasp on analytics to see actual utilization (e.g. number of signups)
  • Have internal metrics as well as external (use Google Analytics) to get a good picture of what users are doing and the conversations about your site.
  • Reaffirmed that cloud computing is a fantastic way to scale super cheaply (AWS/EC2)
  • Automate as much as you can including unit tests, build and deployment scripts, etc. The time savings and reduction in errors pay off quickly.

Become a PersistentFan if you haven’t already, let us know what works/what doesn’t and look for a steady stream of changes. (Any questions why I’m the Mayor at the local Starbucks?)