Hodson Blog

Great post about NoSQL and Big Data in the cloud - an overview that also discusses a portion of the Bing Social Data Platform (I managed this team and the larger platform effort during my time in Bing).

The numbers are quite interesting for scale geeks like me:

It’s also used by the Bing search engine to provide almost-immediate publicly searchable content from Facebook or Twitter posts or status updates. With around 350TB of data, the scope of Facebook and Twitter data is remarkable. When this data is being ingested, transaction throughput reaches peaks of around 40,000 transactions per second and totals between 2 to 3 billion transactions per day.

To summarize:

• 40k trans/sec at peak
• 2 to 3b trans/day
• 350TB of data. The numbers and scale
  

Last month I officially joined the Board of Joint Venture Silicon Valley and am fortunate to be featured in their recent newsletter.

Working at the intersection of the public and private sectors is new to me and I've already learned how little I know about how local government functions. Hopefully I will be able to contribute and help make progress on some near-and-dear quality of life issues like traffic/commuting and the Bay Area environment.

The first Board meeting was quite humbling.

Probably won't be the last time I feel that way.

50,000 runners take up a lot of space

This is a bummer

On April 30th, we will turn off posterous.com and our mobile apps in order to focus 100% of our efforts on Twitter.

Sigh.

After a bunch of procrastination and failed attempts to migrate to Wordpress and tumblr, I checked out Posthaven. Signup was quick, had to agree to $5/month (geez, I better blog more...) and the import was painless.

Good job Gary and company, now when do I get an iPhone app?

Organization – Well organized, marked course. Full marathon transportation was fine but race started late. Grade: B

Course - A few scenic views, spectacular ocean views for last 2.5 miles . Grade: B

Aid-stations - Frequent aid-stations with water that wasn't from a garden hose. No Gatorade. Plenty of volunteers. Grade: B

Swag – Nice medal and dri-fit short sleeve shirt.  Grade: B+

We stayed just a few blocks from the start so the logistics were nice and simple. No bus to catch at 4am, just a short 10 minute walk in the dark. The race started at 6am on the dot. We ran the initial 3 miles through downtown and then hit the first big hill between Miles 3 and 4.

Once we hit Mile 5, the scenery began to change into the beautiful wine country of the Edna Valley. The course was green with a number of vineyards on the way out to Mile 13. There were a lot of rolling hills, some fairly steep.

After a brief out and back for Miles 12 and 13, we headed West and then back towards town. The hills didn't let up! By Mile 20, my calves were screaming and my quads were hoping the rollers were coming to an end.

At Mile 23, we re-entered town and had a net downhill to the finish. There were a few uphills, especially having to climb up to a footbridge to go over the railyard and a sadistic last hill at Mile 25.8 (who does that?!?) The last half mile was all down hill and made for a very fast finish.

Organization – Well organized, marked course. The expected inagural challenges were nowhere to be found. Grade: A

Course - Scenic Edna Valley. Big hills in multiple spots. Grade: B+

Aid-stations - Frequent aid-stations with water that wasn't from a garden hose. No Gatorade. Plenty of volunteers. Grade: B

Swag – Nice medal and dri-fit long sleeve shirt.  Grade: B+

MVC is a great pattern that provides nice abstraction and allows for an easy way to divide tasks amongst a team. Driving site navigation, externalizing connection info, messages and other properties is a great idea and allows for changes to a site that doesn't require recompilation of binaries.

Coupling the above with Tomcat and the ability to hot deploy makes for a great agile team. Things can work in a Continuous Integration/Continous Deployment model. Drop the files in, tell Tomcat to reload and life is good.

Well, almost.

Unfortunately, if you've utilized hot deploy, you find pretty quickly that there are a number of strange OutOfMemoryError exceptions in the logs. They are difficult to reproduce. Something is just not right ...

As I learned last week at the SV Web JUG meeting entitled "Is Instant Redeployment Really Possible", there are a number of reasons that hot deploy doesn't really work and the solutions to making it work aren't easy to accomplish.

Through the use of numerous code examples (and a few Starbucks gift cards), the meeting's guest speaker Sang Shin explained in detail why hot deploy is an idea not ready for primetime.

If your code has statics or singletons (and what code doesn't), the references to these objects are not neatly handled. Enough redploys and boom, you are out of memory. Essentially, the class loader holds a reference to each of the classes it has loaded and any class that can't be unloaded cleanly causes a memory leak.

It turns out to be that simple really - without purchasing one of the various commerical products that purpotedly handle the reference issue (which they do by creating an individual class loader for each and every class, property file, etc) you're hosed.

Not exactly the promise of hot deploy. Which is why I always have n + 1 servers running behind mod_jk

The marathon was small, with about 500 runners and 100 three-person relay teams.. The weather was cool (40F), dry with little wind. As this was a January race in Northern California at an elevation of 1000 feet, I was happy there wasn't snow on the ground.

We got up at 5:30am and took the 7am bus from the finish to the start. The driver didn't get lost and we arrived with plenty of time. The starting line was the top of the Shasta Dam, which turns out to the second largest dam in the United States. They opened the Visitors Center where everyone hung out until race time, which was nice as it was warm! With five minutes to go, they asked us all to head to starting line.

The race began on time and was well-organized. The first 1.5 miles of the course went directly across the top of Shasta Dam and then headed steeply downhill to the river's edge (outflow of the dam) for another mile. Once we got to the level of the river, we ran along a "Rails to Trails" route that followed the river for approximately 23 miles. The views were incredibly beautful and the numbers of marathoners was small enough that it felt more like a nice long run than a race.

We ran through the old train tunnel at Mile 4 (stopping for photos of course) and continued on to the first set of hills at Mile 12. Along the way we were on the lookout for ice on the course - parts of the trail were in the shade and were slippery for the first few hours of the run. Miles 12 - 14 had some hard hills; we also passed the first hand-off for the relay so there were lots of kills (me) after that.

After another mile long hill from 15 to 16, we did an out and back then headed across the Ribbon Bridge at Mile 22.

The last four miles followed along the Sacramento River through downtown Redding and we finished by crossing the Sundial Bridge.

Overall, here's how the run rates in my book:

Organization – Well organized, marked course. Grade: A

Course - Beautiful, remote course. Big hills at Miles12-14. Grade: A-

Aid-stations - Fewer stations than a large marathon, but spaced apart approximately every 2 or 2.5 miles. Well stocked, super friendly volunteers. Water that wasn't from a hose and Gatorade (the Smurf one) that wasn't too strong. Grade: B+

Swag – Great medal, nice dri-fit long sleeve shirt.  Grade: B+

The "new" Google Reader

The "new" GMail

Twitter's Bootstrap site and their commentary on buttons

Is the "Compose" button supposed to tell me when I'm sending mail that is dangerous? I don't get it. Or is the danger here that a designer at Google is actually trying to drive away users?