Why we need to get more comfortable with the unknown

Products and solutions succeed by solving real problems. Creating them can, and perhaps should, seem daunting. Why? Because the challenge isn’t simply having an idea and making it. We need to solve the right problem in the right way, and enough people need to be willing to pay for it, too. Setting out on that journey is intimidating enough, but something else tends to block the path to success.

Problems are always going to seem daunting. So, when we're starting from scratch, we need a destination. But, more often than not, we don't need a blueprint. The teams that struggle most tend to be the ones that are most committed to their initial conception. Building exactly what you envisaged at the outset, without testing it against reality as you go, is one of the most reliable routes to commercial failure.

Building exactly what you envisaged at the outset, without testing it against reality as you go, is one of the most reliable routes to commercial failure.

Richard George

Ways of Working Practice Lead

Before you've invested too much of the budget, you really need to establish whether what you're building is needed. Not just whether it can be done or people like the idea of it, but will enough people pay for it? Those are very different questions.

It's likely that we won't know exactly what we're building – or whether it's the right thing – until some way into the project. That's not a flaw in the process; it's the nature of it. Learning whether something is viable is more important than building it. It's essential to remain open to changing course. However, the flipside of that is we need to spend a significant part of the project admitting we don't have all the answers.

This requires courage – as individuals, as a team and as an organisation. If you can't be honest about what you don't know, it becomes harder to find out. Several factors typically make it difficult to be open about the need to learn, explore and admit we don't know, and these often spring, unwittingly, from internal stakeholder expectations. I'll discuss some of the more common ones below, along with some suggestions for how to mitigate them.

Management expectations

The strategic view of an initiative will inevitably understand the project primarily in terms of the expected result. It's completely reasonable to expect a high-level view of progress so the organisation's wider activities can be orchestrated. However, simply viewing the project as being at some point between 'not done' and 'done' can lead to unrealistic expectations or assumptions that it is just a case of completing the work.

Teams cannot know the unknowable. We must acknowledge that and remember that, if we're genuinely solving a problem, it takes time to gain that knowledge. Teams are often asked seemingly simple questions, such as 'when will this be done?', 'how much revenue will this generate?' or 'how much will it cost to complete?' These are rarely simple questions, certainly when you're creating something new.

A manager may only be looking for an indication, a rough order of magnitude or a range, but by asking the 'simple' question, people often feel pressured or exposed. This can lead to either over-cautious responses that set expectations very low, estimates so optimistic as to be dangerous, or an unwillingness to provide any level of estimation at all. At the sharp end, it can feel like you've been asked to predict the outcome of a sporting event, about which you know very little, or maybe guess the winning numbers in the lottery.

We need to spend a significant part of the project admitting we don't have all the answers. This requires courage – as individuals, as a team and as an organisation.

Richard George

Ways of Working Practice Lead

Managers can help by explaining why they need an estimate, how they are planning to use the information, and what level of effort to put into it. In my experience, a 'finger in the air' is often good enough in the early stages. It's also worth remembering that, whether explicit or not, an estimate should always come in two parts: the estimate itself, and the level of confidence that you have in it. You might, for example, predict, with a very high degree of confidence and very little research, that it will be sunny on any given day in Phoenix, Arizona. However, a similar prediction for London would probably come with a much lower level of confidence attached.

The most accurate way to figure out how long something will take or how much it will cost is to actually do it. Extensive modelling of the outcome will only increase the cost and delay delivery. It may still even be very wide of the mark, since estimates are often initiated when you know least about what's involved.

Team expectations and aptitudes

Simon Wardley (after Robert Cringely) breaks people down into three categories: Pioneers, Settlers and Town Planners. Whilst this might seem quite simplistic, it neatly encapsulates three levels of attitude towards risk and comfort with the unknown. Understanding your team member's risk appetite will help you get the best out of them by aligning them with the level of uncertainty in the work.

If you want to achieve something that is largely unknown and somewhat risky, Pioneers are the type of people you need. They're relatively comfortable with high levels of uncertainty and even thrive in this type of situation.

Conversely, maintaining a mature system, or doing highly structured and predictable work might best be accomplished by Town Planner types. Settlers tend to be used to bridge the gap, operationalising new products and innovations. However, doing this kind of work would likely demotivate Pioneers. The same is true of managers: if you put a Town Planner manager in charge of a Pioneer team, the one thing that you can predict – with a reasonable degree of certainty – is that it will end badly.

Company culture

Companies that tend to only celebrate successes tend to foster quite conservative attitudes to risk. It sends a message and teams and managers can begin to avoid engaging with anything that might be viewed as a failure. At its worst, this can lead to stagnation.

I would argue that the biggest risk is usually not to risk anything at all. If you keep doing what you've always done and avoid the unknown, sooner or later someone else will do it better, cheaper or faster. Or all three.

It's not about celebrating failure either. The path to success lies in being able to admit we don't yet have the answers and accepting that learning is at the heart of the product development journey. The companies that are most likely to prosper, especially in times of rapid change, are the ones that encourage (structured) exploration. The value of failure is measured only by what you learn from it. Learning companies tend to become more resilient and better able to course-correct in turbulent times.

Hiding behind the process

Sometimes people fall into thinking that following the process is the job, when – of course – the outcomes are the job. It can feel safe or comforting when we 'follow the process'. Perhaps, it's because we don't feel like the goal or timescale was realistic, or the expectations (or demands) are too high. Sometimes, it's the result of feeling that the desired outcomes are just too distant from our daily work, so, if we only focus on the next thing to do, the outcomes will somehow take care of themselves.

Creating successful products is a process of finding out as quickly as possible. What should you build? What should it do? Who is it for? Will they use it? Will they pay (enough) for it? You can ask customers what they'd like, but all kinds of biases kick in. If you show them something, they might say "that looks good". If you ask them if they'd pay for it, they might even say "yes". But hypothetical questions yield hypothetical answers. We need to find out.

A better approach is not to ask customers what they want: it's to get them to commit to something. Rather than "would you like this if we sold it to you?", say "we're thinking of doing this. Give us your email address and we'll send you more information when it's ready."

Financial commitment is the best indicator that you've got a good idea. If the right type of people and organisations won't commit, you've got a pretty good indication that you're not on the right track.

Richard George

Ways of Working Practice Lead

Seeking and securing commitment, however small, helps us know we're on the right course. You could extend it to: "Pay £X now and we'll give you a discount when it's ready."

Financial commitment is the best indicator that you've got a good idea. If the right type of people and organisations won't commit, you've got a pretty good indication that you're not on the right track. Painful, but not nearly as painful as finding out after you've built and launched your product.

The teams and organisations that build successful products share one quality: they treat not knowing as the starting point and don't see it as a problem. They seek out data that helps them orientate. Build the minimum, learn the maximum and adjust. The unknowns of product development aren't a distraction, they are the whole point.

Also from Richard George:

Why recognising uncertainty can be a project in itself

Most software projects involve a degree of uncertainty, especially those that deliver competitive advantage. The problem with uncertainty is that it comes in many forms and often surfaces when you least expect it. So, how can teams embrace and navigate uncertainty?

Read blog →