Determining the correct decision in any specific situation requires the ability to look at both qualitative and quantitative data. However, all of this data comes from the past.
For example, if you sold Widget A 5 years ago and it flopped miserably, you are less likely to recommend bringing back Widget A today.
I’ve heard the refrain many times in the past – “we tried that, and it didn’t work” – and look there is a ton of evidence that it didn’t work – market research, studies, customer feedback, etc. But that was then, and this is now. Even a short span of years (especially in this world of unbelievably rapid change) is light years away from today.
Do you have the complete picture of why Widget A failed? Or do you have just enough information to justify your decision to kill Widget A?
For example, I launched my startup AdviceTrader back in 2001. It was an early version of Quora – very similar. We ran that for approximately five years, within which we added the ability for customers to chat with an expert and eventually have calls with an expert. However, after five years, we had an income but had never reached the heights of profitability. A few years later, Quora launched and is now quite successful.
In reality, every moment in time is very different from every other moment in time. If it worked back then it won’t necessarily work today or in the future. Or it could be the exact opposite. The key is to look at the empty spaces in the data and figure out why that happened.
Another example, I was doing a case study in college, and they provided us with a set of both qualitative and quantitative data to work with. We were to use that data to come up with an argument and a conclusion based on the data. They provided a good set of both kinds of data, but then threw in a random element – something that you knew instinctively had some effect on the argument, but could not prove it with the data at hand. The course of action suggested that the solution was A or B, but in reality, the answer should have been “not enough information to make the decision” and to err on the side of your cognitive bias.
If you genuinely think about it – we never really have enough data of any kind to make a perfect decision – there are too many unknowns for humans like us to review in the period we need to make decisions in.
Maybe someday, when we have sophisticated AIs which can look at all the data and develop simulations of the real world, run those simulations and follow every path, we may have better prediction engines. Until then, we humans still have to make decisions with the information at hand; we should always strive to gather as much data as possible. But even then, we, in many cases, have to take a leap of faith and move forward with something which might not be thoroughly vetted.
That’s ok because if you think about it, that’s life.
