Wisdom of Crowds

If you haven’t already read it, I’d like to direct your attention to an absolutely fantastic book called Wisdom of Crowds, by James Surowiecki. In it, Surowiecki argues that the decision-making and predictive power of diverse groups of people greatly exceeds that of most individual “experts”. The book is remarkably comprehensive and convincing, and the case studies in it will inspire and amaze you.

Wisdom of Crowds opens with a nice example: 800 people at a livestock exhibition participated in a contest to guess the weight of a live ox (on display) after slaughter and preparation. Some of the 800 were butchers and so-forth; people who should make a good guess. Many contestants were ordinary people with less “relevant” knowledge. But no expert within the competing pool beat the average guess of the group as a whole, which came within one pound of the true weight (1,197 lbs instead of 1,198).

If you’ve ever wondered why almost nobody can consistently beat the stock market over long periods of time, Surowiecki’s book will help you to understand why, and what you can do to foster the wisdom of crowds within your own organization. And the great thing about Wisdom of Crowds is that it also explains the factors that cause groups (including the stock market as a whole) to falter at key moments. For example, the ’99 market bubble and bust. If you want to avoid the dangers of group-think within your organization, reading this book will help.

I read Wisdom of Crowds back when it was first published a couple of years ago. The reason I’m writing about it now is because I had a conversation with someone who asked me if Blink (written by Malcolm Gladwell) contradicts Wisdom of Crowds. I do not believe that it does. Blink argues that, under the right circumstances, individuals (especially some experts) can make extraordinary accurate judgements by relying on the near-instantaneous, sub-conscious processing capabilities of the mind.

Even if you buy this (and IMO, it isn’t clear from the book that snap judgements are more likely to cause good than harm in any scenario) there isn’t necessarily a contradiction. It is theoretically possible that, in most cases, the snap judgements of individuals within a diverse group may still be (collectively) more accurate the the snap judgement made by a single expert. If you’re interested in exploring this matter further, check out this (all-too-brief) debate between Gladwell and Surowiecki.

Anyway, I think that Wisdom of Crowds should be required reading for every business person and producer in the video game industry. Learning how to rely on your co-workers (and customers, and partners) to help make key sales predictions and development decisions could make the difference between success and failure. And, by the way, you’ll feel 100 times better about living in a democracy (and about humanity in general) after reading this book. 🙂

PS. Now, here’s an interesting question: can the wisdom of crowds be applied to design? Most designers shudder at the very thought of “design by committee”, and perhaps the artistic/individualistic nature of design makes it incompatible with a group approach. But why does it need to be “design by committee?” What about a design approach that is driven by an individual (or very small group) but includes frequent, comprehensive group consultation? When you get right down to it, that’s just another way of saying frequent user testing — something that I (and many others) have been arguing in favor of for a while now.

6 responses to “Wisdom of Crowds

  1. Chris Drouin

    In the realm of design-by-committee, Valve clearly does very well with their “cabal process” – GamaSutra has a nice article about it here: http://www.gamasutra.com/features/19991210/birdwell_01.htm . They create excellent stuff that still has the sort of vision you’d expect from an individual’s creative drive.

  2. PS. Now, here’s an interesting question: can the wisdom of crowds be applied to design?

    I’ve considered this question too but, I have never got around to looking into it in detail. I don’t lean either way, yet. I also think about it more as democratic design. I recenly came across this book … its my first step in solidifying y thinking on the matter: http://web.mit.edu/evhippel/www/books.htm

  3. Chris, Scratchinghead — thanks for the links! I’m travelling right now but I absolutely intend to check these out when I return to Boston.

  4. Scratchinghead – just got a chance to check out the link. I didn’t originally process that it was Prof. Von Hippel. I actually took a class with him, back when I was a grad student. 🙂

    Von Hippel’s theory of lead user innovation is really great, and his books on the subject are totally worth reading (though you can probably skip to the most recent).

    Basically, he has spent the past couple decades trying to prove that the best innovations come from end-users whose needs aren’t being met by corporations. The end-user invents something that meets his/her needs, and it spreads organically to other end-users. Often enough, corporations adopt the innovation after the fact. Von Hippel has documented many cases of this occuring.

    The best example in the video game industry is Counter-Strike — created by users, remarkably successful, and ultimately adopted by Valve.

  5. Hello David. I’ve not made any progress with Von Hippel’s but I did glean all that you wrote about his theories thanks to the Amazon reviewers 🙂 It must have been a good experience to learn from the leading author of this theory?

    I read through the Valve Cabal process and came to the conclusion that its little wonder it takes so long to develop Half Life. I’m surprised at their apparent lack of confidence in design. A strong design process should resolve the same conflicts they wait to see occur when viewing an end user playing their game. This sadly hints at the disease that is ‘subjective design’; where everyone in the industry from the CEO down and across, thinks they can ‘design’ a game. The acceptance of his attitude is the dam that stops the sea of design innovation. I’m not aware of other industries where non designers think they can design. What’s harder to accept is that this attitude pervades in an environment where game design has no clear language in the same way that film or literature has. I digress.

    So, can you apply a democratic process to design within a realistic time-frame? A bottle neck for design … code implementation. Until the design has code support, a designer has no idea if the vision, with all its components will be a success. A strong design process should mitigate some risk, but not all.
    Providing designers with tools that allow them to control and manage the design would be the most important step in allowing democratic success. It should go without saying, but trust, stable pipelines, and a clear, communicated design vision would also be required to support an empowering design methodology.

    Two of the most successful games in recent memory may already have the solution … Burnout & GTA. What do they have in common? Renderware. What else? A set of design rules that can be updated and expanded to create a new game experience without re-inventing the wheel. Whether or not the design process is democratic remains to be seen, but I would guess this is definitely the case with GTA. The imbalance and often lack of common sense with some it its mission design suggests it has many cooks. In the case of GTA, it appears there is no head chef of mission design, something sorely needed to maintain consistency.
    Another example exists but I don’t want to mention the developers name as I’m not clear on how open they are on their process. The developer spent money and time creating a game engine with a set of powerful tools that empower their design. My information may be out of date but its understood they started out with a programmer heavy team with some art support to help sell the concept to publishers. Sometime later the developer had a proprietary engine, not so dissimilar to Renderware. Less than a handful of Programmers continue to maintain the engine and occasionally code a new function to support a designers vision. I’m not sure how the art is integrated but I would imagine it could be outsourced with a two to three person team that drives the art direction.

    So, what else would be required to truly encourage a design led industry? A stable graphical environment, backed by market acceptance, where technology is allowed to plateau so design can lead. It can be argued the market already accepts this if you consider the continued success of the Playstation2. Unfortunately the hardware manufacturers, encouraged by developers obsessed with technological innovation when they should be calling for design innovation, consistently seduce publishers who are willing to allie to the vision, strap on their seatbelts and climb the roller coaster once again.

  6. Pingback: The Bell Curve Scar

Leave a Reply

Your email address will not be published.