In Data Analysis, the Right Answers Come Only From Asking the Right Questions

How decision makers can better adjust to understand one another

Data analysis is about more than complex equations for their own sake. iStock
Headshot of Stephen H. Yu

Even highly-educated analysts struggle in the beginning of their careers. Hiring managers often wonder why analysts with a 4.0 GPA have hard time delivering valuable results. There are many theories and opinions about that, but allow me to share a simple point of view here.

Many young analysts struggle because they don’t know what to do, not because they lack statistical or programmatic skills. It’s because in a school, the professor in charge would provide sets of questions to solve. The work in such cases is really about mathematics and mechanics. In the business world, coming up with the right set of questions is the most challenging part.

When you are sitting in a planning meeting, you would hear lots of “we need this” or “we need that.” But such expressions do not directly translate into tangible goals or problem statements. A goal such as “Let’s improve click-through rates from email campaigns” is relatively clear, but to an analyst, it is just the first sentence in a homework assignment. Improvement from what point? What has been going on? How many types of email campaigns are we talking about? For what type of products and services? Using what kind of offers? How are we supposed to measure the results? And ultimately, what do you want? More clicks, more conversions or more money? What if the answer is “all of the above”? Now, that surely sounds like an open-ended question.

No such thing as a perfect dataset

The second reason why a novice analyst struggles is because in real life, there is no such thing as a perfect dataset. For a school assignment, the professor would provide a pristine set of test data on a silver platter. In business settings, datasets are filled with missing, invalid or inconsistent values coming from all kinds of sources. That is why analysts often want minimum two weeks to finish a seemingly simple task. They are buying time, because they know they are the ones who must fix all the mistakes that are made to the dataset before it got to them. In reality, more than 80% of analysts’ time goes into fixing and standardizing provided data. I’m sure they didn’t get a degree in statistics to clean the data all day long, but that is the reality.

One of the prerequisites of being a modern-day data scientist is consulting ability with keen business acumen, along with excellent communication skills. They should get a grip of the situation fast, and translate vague and unclear statements into firm project specifications. Only then, they will be able to squeeze monetary values out of compiled data.

I’ve seen countless cases where analysts built complex models just because it is something that they are good at. When I ask them why they built such models, the ones whose work holds potential at least try to cite business benefits. Hopeless ones just list a long series of technical terms and brag about their mathematical journeys. To end-users who must show results in dollars and cents, minutia of modeling techniques means nothing. Some fancy machine learning techniques do not bring in cash on their own; appropriately applying them to business cases does.

That doesn’t mean that the burden of constructing problem statements and project plans solely falls onto data scientists. In this day and age, all decision makers must have some basic understanding of analytics options, capabilities and limitations. Imagine being a good patient in front of a medical doctor.

How decision makers can help

Stephen is president and chief consultant at Willow Data Strategy.