Working Without A Crystal Ball

Note: I think this post may be interesting to anyone creating, investing in, or distributing games (regardless of whether or not they are Xbox Live Arcade games.) However, I needed to ramble through some seemingly tangential stuff to make my point. Please bear with me. 🙂

XBLA portfolio management is a complex thing… I’m one part cat-herder, one part traffic cop, one part talent scout, and one part “quality control.” (The latter part is especially tricky… who wants to be the guy who turned down Katamari because “the art was weak”, or one of the eight publishers who turned down Harry Potter because “the writing could use polish.”) I approach these roles with a healthy dose of humility (and even anxiety), knowing that at any moment I could become “the moron who turned down [fill in the blank].” Unfortunately, the longer I hold this position, the more likely that becomes!

Trying not to be a moron

So I’ve put systems in place to hopefully help reduce the risk of my own tastes (or lack of vision) from polluting the portfolio. I can’t really discuss the details, but they include a sort of “wisdom of crowds” feedback loop, in which indie submissions are screened and rated by a group of my colleagues within Microsoft (who are asked NOT to discuss the submissions with each other before rating them — mainly to avoid group-think.) The wisdom of crowds can make my forecasts more accurate, and it can help compensate for any subconscious biases I have. Unfortunately, what I don’t believe it can do is help me identify future mega-hits (i.e. “the next Geo Wars“.)

Our marketing team can’t help with that, either. But first (for all you marketing haters out there), let the record show that XBLA’s two most experienced marketing people have regularly out-predicted the rest of the “experts” on the team (myself included) when it comes to sales of upcoming titles. Unfortunately, they stumble where everyone else does… attempting to predict the truly big winners. Were my job to simply build a portfolio of generally-successful revenue-earners, I would hand over the reins to marketing and go on extended vacation. But Microsoft needs more than that… it needs a diverse portfolio that appeals to consumers of all types, innovative content that (hopefully) inspires the industry, appealing IP, and last but not least, mega-hits that encourage consumers to buy more Xbox 360s!

Luck vs. Madden

Unfortunately, I simply don’t believe it’s possible to (reliably) predict original mega-hits, which is precisely why companies like EA have focused so strongly on franchises like Madden and Harry Potter in the past. Riding a mega-hit is far safer than trying to predict one. There are boatloads of research that back up this assertion; the most recent report that caught my eye was in my articles of interest a few months ago. A quote from that report:

Intrinsic “quality,” which we measured in terms of a song’s popularity in the independent condition, did help to explain success in the social-influence condition. When we added up downloads across all eight social-influence worlds, “good” songs had higher market share, on average, than “bad” ones. But the impact of a listener’s own reactions is easily overwhelmed by his or her reactions to others. The song “Lockdown,” by 52metro, for example, ranked 26th out of 48 in quality; yet it was the No. 1 song in one social-influence world, and 40th in another.

Music, like games, can be filtered by objective criteria that nearly everyone agrees upon (for example, I hereby bet that a song consisting of primarily nails on a chalkboard and cats screaming will never become a commercial success.) But once you get past those basic principles, marketing and public whim rule the day.

Improving your odds (or, “how to do something other than Madden”)

So, if you believe all this to be true, what’s the solution? One obvious answer: more content! Place more bets, even if it means spending less per bet. (Here’s an HBR article, focused on the research I referred to earlier, which argues for just this.)

For most web-based portals, this is a no-brainer. Pour on the content, then use various ranking and merchandising methodologies to elevate the most popular stuff. For XBLA, the issue is more complicated. As I’ve written in the past, we’re trying to foster a rapidly growing (but still emerging) ecosystem, which (in part) means keeping content flow at reasonable levels while we improve the Marketplace platform, and while our audience grows. We also have a dedicated team of people who help indies get their content onto the service, and that team can only be stretched so far. So, for the near term, we optimize on portfolio breadth and generally avoid very similar games. Of course, clones exist precisely because predicting original hits is so difficult, so avoiding them is a double-edged sword for XBLA and for developers. (Furthermore, many consumers appreciate minor variants of a game mechanic that they know and love.) In the long-term, there are several potential solutions to these challenges, but that’s a topic for another day.

What about developers?

Of course, “making many bets” is easy for XBLA. I manage a growing portfolio of many games. And thanks to XNA, I have an increasingly powerful mechanism via which to identify hits… a great community of developers who are creating and filtering them for me! See the Dream Build Play contest winners for examples.

Independent developers do not have this luxury; one or two bad calls and they may very well go bankrupt. So what should they do? One possibility is a variant of “making many bets” — quickly making many prototypes and sharing them publicly. Of course, this isn’t an original notion; many developers have bought into “quick prototype” development methodologies (though far fewer have been bold enough to exhibit their prototypes. I’ve personally been enjoying Petri Purho’s public experiments.)

By revealing their prototypes, developers can get invaluable feedback from the community and an early read on the likely success of the game. Furthermore, they can turn the community feedback loop into a buzz engine that drives tremendous hype for the game. Concerns about “giving away the farm to competitors,” while well-placed, are ultimately not worth abandoning this strategy over. After all, any veteran of this industry knows that execution is 99% of what makes a game successful. Ideas are cheap.

Wrapping up

Nobody likes to face the notion that, no matter how brilliant or creative you are, guaranteeing a hit in this industry (without established IP) is usually impossible. Well, unless you’re Blizzard, but I’m pretty sure they’re inhuman, have signed a contract in blood, or something equally otherworldly.

I face this challenge every day. My solution is to harness the wisdom of my colleagues, to increase the quantity of content on our service as quickly as possible (without jeopardizing the ecosystem), and to help foster the growth of the XNA community. These are imperfect solutions, but they’re the best available to me at this moment in time.

Publishers can also do well by placing many (small) bets across a wide area, while (as always) relying on established IP to help stabilize revenues. But independent developers need to find their own solutions. If quickly creating and sharing prototypes doesn’t work for you, find something else that does. Hoping to be “the next Blizzard” is great, but 99% of developers won’t be the next Blizzard. In the off-chance that your company falls into that 99% bucket, having a plan B could help.