Developing new and improved products and services is an interesting problem space. Part of what makes it interesting is the presence of risks, uncertainties, and the possibility of Black Swan events.
For some category of risk, we can calculate the probability. The roll of dice. A roulette wheel. The lottery. Because we can calculate the probability of all possible outcomes we can calculate the economic risk. In some cases the full picture of risks can get very complicated, with many variables and non-linear relationships to take account of. But to be a risk (in the Knightian sense of the word), the information exists that we need to calculate the probability. In these situations, doing an ex-ante analysis and appraisal before taking action makes sense. Of course, we don’t know the outcome, but the possibilities are well-defined, and the distribution of probabilities is also known.
Embracing Uncertainty
Uncertainty is a different beast. We can usually envisage what might happen, but we can only estimate the probability. England winning the World Cup might be a good example of the difference. Movements on the stock market is another. We can perform Monte Carlo simulations to model the interactions to help understand the potential outcomes, but these simulations are only as good as the inputs to the model. Not only are the outcomes not predictable, we can really only speculate as to the likelihood of different outcomes. Whilst it is trivial to imagine the possibility of the market moving up or down, we do not know the probability of the direction and even less about the magnitude. Confidence is an apt word to use here, since our estimates of the likelihood of an outcome rely to some extent on how strongly we choose to believe. Others may have different opinions about the likelihood.
Part of the uncertainty comes from what we might call “complexity”. In particular, we lack knowledge about the circular references and feedback loops — where cause and effect are interrelated in ways we don’t really understand. Even if we did manage to spot some link between cause and effect, if we act on that we become part of the system, changing the system in ways we can only speculate about.
It’s not completely dark in the world of uncertainty and complexity though. Over time, given sufficient experience we start to recognise patterns. The human brain is effectively a pattern recognition machine. Chartists refer to things like “head and shoulders” for the seemingly random movement of stock prices, suggesting historical movements and patterns as a guide for the future. We are so good at seeing patterns, that we even tend to see them where they don’t really exist.
We can also tap into the so called “wisdom of the crowd” to provide some guide (hello, index trackers). But the limitations of these hacks on uncertainty should be obvious. The truth is, these methods of prediction are really more like guides or heuristics rather than anything deterministic. There is a danger in employing them: with every example that works we may gain ever more confidence, believing them to be infallible. The trap here is epistemic arrogance, which leads to foolishness and can expose us to catastrophic failure.
Value is rare, extreme and obvious in hindsight
Which brings us nicely onto a rather special form of uncertainty: Black Swan events. These are a form of “unknown unknowns” that produce extreme results. One (positive) example of this might be the feature request to make a small change to the SAP invoicing system at a global logistics company. It was only after considering the value and urgency of the feature that it was realised that this small change was estimated to be worth as much as $230,000 per week.
The problem with Black Swans is that our dominant mental models are initially blind to the possibility. With only a single observation, our previous mental model is proven wrong and rendered useless. All that “learning by doing”, empirical data, “practice beats theory” goes out the window. Whilst we tend to quickly create a coherent logical story after the fact to explain the (now obvious) result, the revealing of the truth is usually a surprise to us.
Perhaps one of the most extreme examples of a positive Black Swan in product development would be SMS. No one in their right mind would have predicted that the wasted space in the transport header used to locate your phone would be worth trillions of dollars. Perhaps not to the same extent, but somewhere in the backlog of ideas in your organisation there are a few relatively small changes for which the value will be extreme, but only obvious in hindsight.
Projects don’t help
I have written elsewhere about the problem of projects. Projects are a reasonable solution for managing known risks. Plan the work, work the plan. PRINCE2, “projects in controlled environments” and all the other project management methodologies focus on the three critical constraints of cost, time and scope – not value, speed or feedback. They are a suboptimal solution for managing in the presence of uncertainty. Even worse though, they are the totally wrong way to manage in the presence of Black Swan events.
So, what can we do about this? It can be tempting to oversimplify the presence of black swan events, seeing it as simply an uncontrollable aspect of random events or even ordained by some invisible force. “Shit Happens”, “Inshallah”, “C’est la vie” — so we should just do the best we can and maintain a blissful ignorance until the inevitable occurs. Such a nihilistic response, abdicating responsibility mischaracterises the nature of Black Swan Events — and seriously underestimates our ability to do something about it. Just because we are blind doesn’t mean we should run around in a dark room that we know contains knives. Understanding the possibility of black swans actually provides an opportunity to improve the payoff asymmetry in our favour.
The wrong approach would be to test the water with both feet. That would be like using a formal project vehicle where the work is hierarchically decomposed up-front and delivered in a single large batch via a stage-gated process. A more sensible alternative would be to limit our downside by limiting the size of the batch or increment. The whole point of doing this is to gain more information about the likelihood of the option being a beneficial one. To tilt the playing field a little, it helps to make a series of small investments, limiting our downside.
We can also improve our ability to quickly respond by creating and listening to nested feedback loops. This not only helps us to correct for decisions that turn out to be less fruitful (or more painful) than we hoped, but also to chase further and push harder down paths that turn out to be more fruitful than we imagined. This is the “Build, Measure, Learn” of the Lean Startup Method. Faster feedback loops help tilt the playing field a little further, by cutting off poor options earlier.
Another area to consider is how we prioritise (or schedule) and incrementally break down our options. The fuzzy front end of product development is by far the cheapest place to improve time to market. It is remarkably easy to speed this up, and it doesn’t cost a thing – other than a willingness to challenge the status quo and try a different approach. Prioritisation also seems to be a good place to tilt the playing field in your favour. I find it intriguing that we think it’s perfectly justified to ignore economics and go with gut feel or, more usually the HiPPO (Highest Paid Person’s Opinion). Whilst some HiPPOs tend to do quite well here, I’d suggest these are the exceptions that prove the rule. Far better to use an economic framework to help us make these decisions, and quickly decide which seams are more likely to contain the diamonds we are looking for. A little effort with prioritisation not only brings the fuzzy front end into focus, it also speeds up the end to end delivery and the most critical feedback loop of them all: from Idea to Information.
Black Swan Farming: a different focus
The problem is, optimising for these things requires a completely different focus than what we typically find in product development or software projects. Over the last twenty years organisations have become addicted to the false promise that the best way to control software delivery is the project vehicle. We find ourselves optimising for efficiency and making promises to deliver a specified scope on a specific date and to a specified budget. When it doesn’t work, we blame the team, we blame the lack of time and effort planning and “managing risks”. Are we mad, doing the same thing over and over again, but expecting different results? Instead, we should be focused on discovering, nuturing and speeding up the delivery of value. Our mission should be to limit our downside and maximise the opportunity to capitalise on positive black swan events.