Data Driven Delusions

“we just need more data”. “I need another new report”.

Are these familiar sounds in your office environment? Or perhaps you have found these words coming from your mouth…maybe more than once this week!

It is tempting to fall into this trap. The delusion that if you are unsure or are feeling pressure to perform that you must simply need more data and this will be magically illuminating. Not so. If anything this suggests a cry for help and is a major source of frustration for those around you. Especially for those who you’ve just asked for the 5th modified report in as many days.

The famous axiom “a map will not help you if you don’t know where you are today” is something to take to heart if you find yourself on either side of this problem. Instead of rushing to the next published version of a report it may be more helpful to stop, take a deep breath, and ask yourself a few simple questions:

  • What am I trying to achieve?
  • What decisions do I need to make to do this successfully?
  • What information will help me do that?
  • Do I have this?
  • How much work is involved in getting that information from here?
  • Is that much effort worth it?
  • What assumptions must hold true for this all to work?
  • Do I know that they hold true here and now?

A simple set of information that helps you answer important questions is almost universally better than hundreds of pages of numbers and pretty graphs.

Sometimes the things you think are easy to do (like “just add one more column”) to a report might actually take several days and some sort of ritual sacrifice to the gods of IT in order to achieve. Your assumption of simplicity may not hold true. It’s always best to ask and challenge than just pretend you are the Captain of the Enterprise and give the directive “Make it So”. Your team will thank you for it and likely offer up solutions that you hadn’t considered.

Another common issue is the pre-commitment. There is a great phrase in the analytics world…”Some managers use reports and graphs like a drunk uses a lamppost….more for support than illumination”. Simply put, sometimes commitments and promises get made before then numbers come in. Those managers then want to “massage” the data to make it fit their original assumption or story. If this is you, you might find employing an analyst less beneficial in the short term than simply using Excel and “Goal Seek” as your reporting method. The problem is, you will eventually get caught out.

When it comes to data it is always best to measure first, then analyse before rushing to improvements or control measures (Those DMAIC Six Sigma people out there are probably high fiving someone at this…but, like me, a bit socially awkward and excitable when talking stats!).

It is easy to shoot the messenger and blame them because the numbers didn’t support your assumptions or promises to the boss. Information is helpful, but only if it answers the right questions, and that you also fully appreciate what degree of confidence you can have in those answers. If in doubt, or you simply don’t have the time to make sure, seek out someone who can work that out for you. Remember, nobody, including the boss expects you to have to have all of the answers right away.

A new approach without data is just hope. The approach simply seeking more and more data to support your original view is not new, but it is rarely successful.

The right information helps to make sure that your new approach or decision makes sense and has a good chance of success.

Posted in

Nathan Jones