People working on IT and business change projects spend a lot of time deciding what to build or change. Many, many ideas are generated and many, many decisions are made during the course of an average project. But one thing I’ve noticed is that we don’t generally dedicate enough time to identifying alternative options.
I’ve done a bit of research, and it turns out that there are specific reasons for this – it’s to do with the way the human mind works – how we generate ideas and how we make decisions.
In the first half of this article, I explain what I’ve learned about the way we think and act, and what that means for our approach to decision making. This includes quite a bit of theory, but please bear with me – it’s really useful background.
In the second half, I introduce the concept of options engineering – the process of consciously considering options – and how it can improve outcomes on IT and business change projects. Options engineering is easy to understand but hard to do well. So I’ve included 14 top tips on how to become a master options engineer, plus a handy infographic to print out and / or show your friends and colleagues.
Humans and the March of Progress
One thing that sets Homo sapiens apart from the rest of the animal kingdom is the extent to which we have changed our environment to suit our needs. Birds build nests; otters build dams; we build skyscrapers, power stations, nuclear weapons, MRI scanners, nations, belief systems and iPhones.
According to Yuval Noah Harari’s excellent book Sapiens: A Brief History of Humankind, our global dominance started around 70,000 years ago, when we underwent a cognitive revolution, improving our ability to learn, remember and communicate.
Progress was initially slow – for just under 60,000 years we continued to live as nomads, and our growth was limited by the amount of food we could hunt and gather. Then, around 12,000 years ago, we had an idea – we would make the food come to us – and so we cultivated plants and domesticated animals. That was the agricultural revolution, and it allowed us to set down roots and live and co-operate in larger groups. It represented a step change in our rate of growth.
In time, other inventions followed: metal smelting, the wheel, money, writing, paper, social order, government and taxes, the rule of law, false teeth and so on. Around 500 years ago, the pace of change increased yet again with the scientific revolution. Since then, our thirst for change has become insatiable, to the point where we are constantly striving to improve our lot.
Why So Much Change?
Underpinning all this change are two key human traits: our ability to change and our desire to change.
All species have the ability to change, but the only mechanism available to most of them is evolution. This involves a lot of blind trial and error (i.e. random mutation combined with natural selection), and is thus awfully slow. During the cognitive revolution, Homo Sapiens developed imagination – the ability to think abstractly about things that don’t actually exist. This allowed us to believe in imaginary things such as gods, money, nations and laws, but it also allowed us to have ideas for how things could be changed – we could imagine the future. Once we can imagine a seemingly better future, we can create it.
This enhanced ability to change is of no use without a driver for change. What is it that makes us want to change so much? It seems that we have evolved two deep-rooted emotions that cause this – one is dissatisfaction and the other is desire.
Dissatisfaction is the uncomfortable feeling you get when you observe something that doesn’t seem quite right – an untidy desk, an unwashed cup, an advancing foreign army. The uncomfortable feeling drives you to make a change to make the feeling go away – tidy the desk, wash the cup, call in the troops. Larger challenges call for larger changes – faced with a hundred dirty cups you might purchase (or invent) a dishwasher. Some people are more prone to dissatisfaction than others – my colleague Ed’s desk is tidier than mine.
Sometimes we label the unsatisfactory thing as a problem, and we feel driven to hunt for a solution. But because the feeling of dissatisfaction is a personal thing, calling it a problem is subjective. Ed’s untidy desk is a problem, but mine isn’t. A closed road is a problem for commuters but not for locals enjoying the peace and quiet.
Desire is a more positive emotion – it’s the happy or excited feeling you get when you are inspired to improve your world – a new car, a garden planted with flowers, a wind turbine to generate electricity. It can also be linked with another positive emotion – a sense of purpose – and, once the change is complete, desire and purposefulness are often replaced by yet another positive emotion – the satisfaction of a job well done.
The Nature of Ideas
Whether driven by desire or dissatisfaction, all ideas happen in roughly the same way:
- We observe something
- We feel driven to change it, either by dissatisfaction or desire
- We imagine a future which is somehow better
These three steps might not happen in quick succession. We might observe something for quite some time before we feel driven to change it. Likewise, once we feel driven to change, it can take time (and effort, and collaboration) before we are able to imagine an improved future.
Occasionally, we have an idea without first observing a context for that idea. In these cases we have discovered a “solution” and we then go in search of a “problem” that it might solve!
A key point is that we can only have one idea at a time – we can only imagine one version of an improved future at once; we don’t suddenly imagine several improved futures simultaneously.
And once we’ve had an idea, it’s not uncommon for us to stop there, without really considering whether there are any better options. There are couple of reasons for this.
Reason 1: Satisficing versus Maximising
According to research by Schwartz et al, there are two different decision-making styles:
- Maximising means considering all possible options and choosing the best option
- Satisficing means finding a single option that fulfils the desired outcome without trawling through all the options
Many of us, it would seem, are satisficers – once we have had an idea that appears to fit the bill, we don’t feel the need to look any further. Coming up with further ideas takes time and effort. Evaluating options is hard work. Why bother when we already have an idea that’s “good enough”?
This is especially common if the decision-maker isn’t the budget-holder. We find an idea that works without stopping to consider whether there’s a cheaper option that gives better value for money. My daughter will generally choose a hot chocolate with whipped cream and marshmallows if she knows I’m paying, but not if she has to spend her own pocket money on it.
Reason 2: Emotional Attachment
We get attached to our ideas. A challenge to our idea feels like a challenge to ourselves. This is, of course, the ego speaking, but it can be a strong force in decision-making, especially where the same person is both the idea-generator and the decision-maker.
Also, the more we invest in an idea, the harder it is for us to give it up. To do so would be to admit that we had made an error of judgement. Politicians are notorious for refusing to change their mind on a decision for fear of losing face. Choosing the right idea early is therefore a good thing.
Not All Ideas are Good Ideas
People have lots of ideas. Not all of them are good ideas. I’m defining a good idea here as one where the benefits of the idea outweigh the costs.
People don’t always take the costs into account when suggesting ideas. The most obvious cost is the cost of implementing the idea. Organizational change doesn’t happen by itself – there is always a cost, whether it be new equipment, software development or training. Sometimes there are less obvious costs – I can reduce call lengths at my contact centre by using voice recognition software, but probably to the detriment of customer satisfaction.
If the costs of a change outweigh the benefits, then it’s a bad idea. In Sapiens, Harari argues that the agricultural revolution was not necessarily a good idea – despite significant population growth, people’s lives were harder, more disease-ridden and less varied than they were as hunter gatherers. Whether you agree with him or not, it illustrates the point.
Even if an idea is a good idea, it isn’t necessarily the best idea. The best idea for a given context is the one that gives the most benefit for the least cost. It’s highly possible that the first idea you came up with is not the best one. Other, better ideas are lurking out there waiting to be thought up.
Of course it’s not always possible to measure how good an idea is. Many benefits and costs are qualitative, not quantitative, and the overall goodness of an idea is thus a subjective matter.
But the general principle is: there are always options, including the option to do nothing. As the old saying goes: there are many ways to skin a cat. And it might not even be worth the bother – how much do you really need a skinned cat?
A good way to decide the best option for a given context is to have a competition. Each competitor submits their idea for change and, ideally, tries it out. The best option (according to some agreed criteria) is the winner and is selected.
One example of such a competition is evolution by natural selection. This is not an organised competition; it just happens due to the laws of nature. Life forms of a given species reproduce with random changes known as mutations. Some of the random changes will result in an improved chance of survival. The fittest ones survive to the next generation. The losers do not survive.
Another example of a competition is the free market economy. Companies offer products and services. Consumers decide which ones to buy. The companies with the best (most popular) products and services survive. Companies with poor (least popular) products or services go out of business.
The public sector equivalent of the free market economy is democracy. Politicians offer policies and promises, and citizens vote for the policies and promises they most like the sound of. The winner gets to run the country.
As history has shown, competitions like these can be very effective – they have played a large part in human change to date. But we can do better. Rather than using the first idea we thought of and relying on the market or democracy to choose the best option, we can make an effort to look at the options before we enter the competition.
Options engineering, then, is simply the process of consciously considering options for a given change context. The purpose is to come up with the best possible idea we can think of, rather than implicitly selecting the first idea we thought of.
Here’s how it works:
- Understand the context for the change – the “as is”
- Come up with at least two options
- Select the best option
It’s actually not that hard to do, especially if you follow the tips I provide below.
Tip 1: Remember To Do It!
The first and biggest hurdle is to remember to consider options in the first place. It’s all too easy to take a proposed change at face value and just get on and implement it.
For this reason, I try to follow a three-step process when I design a business or IT change. It looks like this:
The important step is Define – after a change has been requested but before I delve into detailed design, I make a conscious effort to consider options. There are some other things I do during Define too, which we’ll cover as we go through the various tips.
I’m a big fan of agile software delivery. In agile, we represent the scope of our change as a set of user stories which we capture and prioritise on a product backlog.
When working in an agile environment, I try to follow the above three-step process for every single user story.
The exit criteria for Define are simple:
- Options considered
- Preferred option selected
Sometimes I call this step Discovery, but it is not to be confused with the focused discovery that sometimes happens during project inception – this is discovery on every single user story.
To make sure we don’t forget the Define step, we even have a separate column for it on our team’s task board:
Including this step for every story might sound like a process overhead, but in practice it isn’t. If a story is already well-defined and it’s obvious that considering options won’t add any value, the story sails through Define in minutes.
Tip 2: Consider Options at All Levels
Designing change is a hierarchical process, driven by a hierarchy of goals. At the highest level, an overall project goal is defined (for example, “launch an online grocery service”). A number of sub-goals then emerge (“provide a customer shopping portal”, “provide a customer support facility”, “provide logistics”), and these sub-goals beget their own sub-goals with ever-increasing levels of detail until you get to the point of being able to build software (“allow the user to enter their credit card number”).
Agile supports this goal hierarchy via the discipline of story splitting, which is also a hierarchical endeavour.
Options exist at all levels in the story hierarchy. At the very highest level, “provide a web site” and “provide a mobile app” are two options to fulfil the goal “provide a customer shopping portal”. At a much lower level “12 point font” and “10 point font” are options for “allow the user to enter their credit card number”).
Therefore, I do discovery on every story, regardless of what level it is at. The types of options are different at the various levels in the hierarchy, but there will always be options.
The importance of considering options is greater for higher-level stories – the decisions are likely to be more strategic and more difficult to change later. So it makes sense to expend more effort on options for higher-level stories. As ever, it’s a balance.
Also note that working hierarchically allows me to make decisions at the right time. Higher-level (strategic) decisions are made sooner and lower-level (detailed) decisions are made later. All decision are made “just in time”, also known as the last responsible moment (although admittedly this is a controversial term).
We’ll return to story splitting in tip 9 when we look at phasing.
Tip 3: Understand the As Is
As discussed, once we have an idea, we tend to dive in. We forget to consider alternative options. We also forget to do our research. It’s just too tempting to start working on the “to be” without fully understanding the “as is”.
Failing to fully understand the wider context of the “problem” that we’re trying to solve can result in sub-optimal designs and, in some cases, changing something that wasn’t even a problem in the first place when viewed from a different lens.
For example, a goal for a contact centre might be to reduce staff costs by introducing an automated call routing system (“press 1 for this, press 2 for that”). But some user research might unearth the fact that many customers chose this provider because they prefer to speak to a human.
Another example: recently I implemented a service for accountants that aggregated their client’s tax and income into one easy-to-use total. But only later did we discover that the accountant needs to check the individual figures before adding them up.
In both cases, user research on the as-is process would have helped us choose a better option.
This is, of course, basic stuff, but it’s all too easy to forget in the midst of a “good idea”. Classic techniques include:
- Sitting with users and observing them doing their job
- Interviewing users about how they do their job
- Playing with, or actually using, the IT systems for yourself (either in live or in a test environment)
- Understanding (and maybe drawing) the as-is business process
- Understanding (and maybe drawing) the as-is data model
- Understanding (and almost certainly drawing) the as-is architecture – in a context diagram
Having enough understanding of the as is helps me make better decisions about the to be. It ensures that new processes and functions fit in well with existing ones, and improves overall consistency.
So, do some research, make some notes, draw some diagrams.
Tip 4: Understand the Objective
Having understood the as is, the next question is: what are we actually trying to achieve? Answering this question is harder than you think because objectives cascade in an infinite regress. I can illustrate this by applying the classic 5 whys technique to a seemingly simple problem:
Bob: I want brakes on my car that don’t lock up.
Bob: So that my car doesn’t skid when I brake hard and the road is wet.
Bob: So I have better control of my car when braking hard in wet conditions.
Bob: So I can avoid crashing my car into an obstacle.
Bob: So I don’t die in a horrific car crash.
Which of the above is the true objective? If it is to avoid skidding, then my range of options is limited to things like anti-lock brakes, better rubber on my tyres, driving more slowly or not driving in the wet. But if the objective is to avoid death then I have a wider range of options, such as airbags, crumple zones or working from home.
That said, if I have been splitting user stories, it’s likely I already have a hierarchy of objectives (aka goals), and the objective comes “for free” as part of the standard agile story format:
As a (user or stakeholder)
I can (function)
So that (objective)
The “so that” part of the story is the objective I get by asking “why” just once, which is often enough. If the objective is in doubt Ican of course ask why again, and again, and again.
Once I have agreed the objective, I can ask a very important question:
How else could we achieve (or partly achieve) this objective?
Asking this question opens the door to options.
Tip 5: Start With Two Options
In my experience, the jump from one option to two options is the hardest. As soon as we have two options, further options generally arise, especially when you collaborate (see tip 7 below).
So, I aim to identify at least one alternative to the original idea that’s on the table. It doesn’t even need to be that good, it just needs to be different. Its purpose is to stimulate further ideas.
Tip 6: Visualise the Options
For most other design disciplines, such as building design or product design, visualisation of the design is an intrinsic part of the process. Not so much with business and IT change – we commonly resort to the written word to explain an idea.
Buildings and products are easy to draw because they exist in the physical world. The only real challenges are representing a three-dimensional object on two-dimensional paper and illustrating any moving parts.
Organisations and IT systems are more difficult to draw. Static drawings of how they look are of some use – the layout of products in a retail store; the layout of text and data fields on a computer screen (i.e. a UX mock-up). But most of the interesting aspects aren’t visible to the human eye – the logical arrangement of business units or computer systems; the dynamic behaviour of people and systems over time.
We have developed ways to visualise these things – context diagrams; process flows and so on. We should use them.
Visualising ideas has a double benefit:
- It’s easier to share, and discuss, the idea
- The process of drawing helps the drawer to formulate the idea and spot holes in it
I usually aim to produce one or two visualisations of each option. The type of visualisation I choose depends on the story. If the change is a user interface (UI) change I usually create a low fidelity screen mock-up. If the change is a back-end change, a context diagram often works better. Sometimes a data model helps. I draw the same diagram for the as-is and for each option, keeping the diagrams as similar as possible to each other so it’s possible to see the differences between each option.
Drawing takes time, so I don’t always do it – it depends on how big the decision is – but in the vast majority of cases, a picture is so much easier to discuss when you follow the next tip.
Tip 7: Collaborate
This tip is essential. Because of the way we create ideas, and because many of us are satisficers, and because we tend to get attached to our ideas, the best way to ensure options are considered is to involve more than one person.
Collaboration is fairly standard practice for agile teams nowadays – teams will meet to discuss an upcoming story in a three amigos session. However, it’s normally expected for the product owner or business analyst to already have a good idea of what’s to be built beforehand.
What I’m proposing here is to also use the three amigos session to discuss options. This means collaborating a little earlier than usual – perhaps two sprints in advance instead of one – to allow time to work through the options and make a selection.
Discussing options might require a slightly different audience in the three amigos than usual. I almost certainly want your product owner in the room. I might also invite other key stakeholders if you don’t think they will be too disruptive. Whether it’s still a three amigos session at this point is a good question. Maybe it needs a different name – three optioneers, perhaps, or three exploradores?
Anyway, the general plan is to have a brainstorming session. I explain the objective, the as is and the two or more options you already have. And then I invite suggestions for alternatives.
In a brainstorming session, it’s important not to discount any ideas. At this point we’re looking for options – we’re not going to select an option yet. Even if a suggestion seems inappropriate it’s a good idea to capture it anyway – that keeps everyone involved and engaged, and also you never know when a seemingly bad idea can trigger an absolute peach of an idea!
Tip 8: Consider Non-IT Options
If I have a hammer, everything looks like a nail. If I’m an IT practitioner, every problem can be solved with IT.
Because IT practitioners are so used to designing and building IT systems to solve problems, we tend to think of IT solutions first. It becomes second nature – I’m as guilty of this as the next man.
This is tunnel vision again. So, I try to make a point of considering at least one option that involves no IT change. Sometimes we refer to this as a “manual workaround”, implying that we would use IT if only we could, but we can’t. In truth, sometimes a manual process is the best option. IT systems are good at doing simple repetitive tasks quickly and cheaply. Humans are good at doing more complex tasks, but are slower and more expensive. If the task at hand is complex and infrequent, a manual solution might be most appropriate.
Whether the manual option is best will become apparent later when we look at pros and cons.
Tip 9: Consider Tactical Options and/or Phasing
A common approach to design is to think “what would a really good solution look like?” Often there’s a solution which is strategic, fits well or just “feels right”. But sometimes the “right” solution is also an expensive solution, because it deals with every conceivable scenario.
Therefore, I find it useful to ask a couple of questions:
- Is there are simpler solution which is still “good enough”? Maybe it handles the most common scenarios automatically whilst leaving more complex scenarios to be handled manually.
- Is there a way we could phase the delivery so we do something simple first and then work towards the better solution? Put another way, phase 1 is the tactical solution, which we would do sooner, and phase 2 is the strategic solution, which we might choose to do later if we have other higher value things to do first.
This approach goes hand-in-hand with story splitting – each phase becomes a separate story, which can be prioritised separately on the backlog. It’s also very similar to the Good-Better-Best game described by Jeff Patton in his book User Story Mapping.
Tip 10: Consider Doing Nothing
I said earlier that ideas generally start with an observation of something and a desire to improve it. I also said that not all ideas are good ideas – the costs of the idea might outweigh the benefits. It might be that there is a better idea, but in some cases it might be that none of the ideas we can think of for a given context result in an overall improvement.
So I always consider the “do nothing” option, also known as “leave well alone”. Life is one big compromise – it’s not possible to make life perfect in every respect. More staff in the call centre mean shorter waiting times and/or more time with customers, but higher costs. Fewer staff means lower costs and happier shareholders, but unhappier customers. It’s a balance. Maybe we already have the right balance and nothing needs changing.
Tip 11: Determine Pros and Cons
Once we have collaborated to generate a whole bunch of options, it’s important to compare them. Over the years I’ve read about all sorts of techniques for carefully and scientifically comparing options, involving a variety of complex calculations to arrive at the perfect answer.
But I find that usually, simply listing the pros and cons for each option, and having a discussion with the team, is good enough, and a whole lot quicker and simpler.
Of course, one of the cons for each option is the cost of delivery. Often, we’ll be choosing between a “better” but more expensive option and a “less good” cheaper option. So it’s a really good idea to estimate the effort for each option. If my team uses story points for estimating, we can use then for estimating options too. The estimates don’t need to be that accurate at this stage – just enough to be able to select an option.
Tip 12: Sleep On It and Iterate
Ideas take time to come. Especially the good ones. People often talk about having an “a-ha” moment randomly and without warning. So when working on options it’s a good idea to allow them to gestate for a short while. Time is, of course, money, and we can’t wait too long. But certainly for bigger pieces of work, if I can plan in at least two separate collaboration sessions with a day or two between them (or even longer), this gives us all the opportunity to mull things over and maybe have an “a ha” moment.
Good ideas come at the strangest of times. Apparently it’s quite common to have them in the shower. I had an “a ha” moment last week whilst out walking the dog.
Taking a little time also allows us to iterate and refine. I sometimes run several team brainstorming sessions on the same user story, with time in between for me to elaborate the various options and do more research. My new research might generate yet more ideas and so we loop round again.
Tip 13: Manage Your Energy
There’s a balance to be struck between maximising and satisficing – I try to make sure the effort and time we spend on options engineering doesn’t outweigh the benefit – especially the opportunity cost – doing something sooner sometimes beats doing the best thing later.
So, I try to realise when we have something that’s good enough, and I try not to sweat too much if I think we haven’t fully maximised. The 80:20 rule applies as much to options engineering as it does to the product backlog overall.
Tip 14: Remember to Test Your Hypothesis
Mature agile teams know that the only way to discover whether an idea is a good one is to test it with real users. Put another way, any idea we might have, even if we have considered options and selected the “best” option, is a hypothesis for an improved future.
There’s a strong parallel with the scientific method here. Scientists generate hypotheses for how the world works, and then conduct experiments, or tests, to either confirm or refute those hypotheses.
So it’s a good idea to test our idea to double-check it’s a good idea.
We can test your design at various points in the delivery lifecycle. The earlier we test, the cheaper it is to alter or abandon the idea, but the less accurate is our testing.
Here are some common approaches to hypothesis testing:
- Show the design to stakeholders and/or users – this is the usual approach – review the design on paper and invite feedback
- Build a prototype and get real users to try using it before we build anything more
- Measure the benefit of our change once it has gone live
Pretty much all teams use approach 1. Many teams use approaches 1 and 2. The very best teams use all three approaches in combination, and iterate their solutions based on real feedback from real users using the real system with live data. Teams that live and breathe this approach are practising Hypothesis-Driven Development.
The implication is that we won’t necessarily get it right first time, even if we do practise options engineering and follow all of the tips above.
Learning to accept failure and go back to the drawing board is probably the hardest skill of all!
We humans have an innate desire to improve our environment. We have lots of ideas for how to do this, but we sometimes get attached to our ideas and suffer from tunnel vision, even though not all of our ideas are good ones.
That’s a lot of tips! But don’t panic – you don’t have to follow them all to gain some benefit. Even if you only follow the first tip – actually remembering to do options engineering – you should gain some benefit. Everything else is just the icing on the cake.
If it helps, you could print out the infographic and put it up on the wall as an information radiator and /or share it with your team.
I call this activity options engineering as I feel this phrase conveys the structured approach to identifying and selecting options. If you prefer your techniques to have a slightly more funky feel, you could always call it optioneering, which is apparently already a thing.
Options engineering is also complementary to Feature Injection, which is a different way of thinking about identifying value and deciding what to build.
And if you want to learn how to put what you’ve just read into practice, you might want to look at my Distance Learning Course.