Why Retention Stats Often Give You Zero Insight Into the Customer Experience

Not long ago, I was sitting in a meeting discussing the profit-and-loss (P&L) statement for one of our new product lines. The P&L manager was running through the key performance indicators (KPIs), and we got to the churn metric, when she extolled, “Churn is holding flat, so we are looking OK on that front.” Everyone in the room nodded their heads, and then we started to move on to other parts of the agenda. At that moment, I internally debated whether to let the comment go or to interject. I just couldn’t help myself.

“Sorry to bring this back, but I think we might need to look a bit closer at our churn. If our base churn is holding steady, it means we are actually getting worse at retaining our customers.”

Everyone looked at me with befuddled faces. Are you equally confused? A lot of digital advertisers just haven’t been taught the right concepts, therefore let me explain.

Base churn is the ratio of customers you lose in any given month relative to the size of your customer base. If you have a customer base of 100 and 10 stop using your product or service, then your base churn is 10 percent. The problem is that, in truth for the vast majority of companies, your new customers churn at a much higher rate than your existing customers. This is because many of your new customers are just giving it a try.

In the early days of your product, your new customers make up a high proportion of your customer base; so your average churn looks really high. But over time, new customers make up a smaller and smaller proportion of your customer base. And as a result, your average churn should decline over time. Therefore, if your base churn is holding steady, it’s highly likely in reality you have a churn problem.

So what’s the best way to measure churn? Here’s a five-stage outline that I’ve created:

Stage 1: Evaluating base churn

This is where most companies start. And as I mentioned above, at a basic level, it is a simple ratio of your lost customers to your base customers. See the calculation below.

You can see that the new customer churn is holding steady at 50 percent while the existing customer churn is standing flat at 1 percent. But as the existing customers become a greater proportion of our customer base, the base churn percentage declines. The other issue with this metric is that it gives you zero insight into the customer experience. It’s a dumb outcome metric.

Stage 2: Calculating cohort churn and graduation rates

To truly get to grips with churn, you need to look at KPIs that give a much better insight into how customers are really feeling about your product. So let’s look at cohort churn analysis, or simply “cohort” for short, which compares a customer base on when they became customers and shows where in the journey major attrition events occur. An obvious evolution of the above base churn calculation is to look at each acquired cohort separately.

If you look at your January cohort distinctly from your February cohort, you can now track any given customer cohorts churn over time. Reformatting the above table, you would then see something like this:

You can see now that we can compare each cohort’s churn rate month over month. We would see that for each cohort we retain 50 percent into the second month and then 10 percent into the third month. We are now comfortable that our churn rate is holding.

One flaw with this method is that it is looking at the data from a month-by-month perspective. The problem is that in one month, let’s say March, you acquire all your customers on the last day of the month then the churn into April is going to look great.

To solve for this issue, what you should really do is look at your churn on a 30-day, fixed basis, focusing on how many customers were still active after 30 days. We are presently getting into ninja territory, but you can now measure the percentage of your customers who made it to day 30, day 60, day 90, etc. These are called graduation rates, and they represent what percentage of your customers made it through 30 days.

Stage 3: Measuring graduation rates

The idea here is that the churn metric you should consider is what percentage of people are still with you after a set period of time. You can look at the percentage of people over time who made it past 30 days, 60 days and so on. Most product churn happens in the first 90 days, so starting out, be hyper-focused on early life graduation.

Is your 30-day graduation rate improving? If so, are there any similarities in the way these people are using your product versus those who’ve churned? If so, try to proactively educate the customer to use these parts of your product.

Stage 4: Zeroing in on those who reach “Aha”

This measurement, to me, is the best way to weigh churn and to impact it. It is a relatively common concept in more-established business like telecoms (where I started my career), but not so much with digital advertisers. When customers sign up to use your product they often have to take some actions and see some results before they truly understand the value your product brings.

You need to figure out what combination of product feature usage or stage within your product a customer needs to get to in order to experience that “Aha” moment, which is when they get the value of your product not because you’ve explained it but because they’ve experienced it. For some customers, it will take only one day while for others it will take longer. It all depends on their budget and how they set up their campaigns. This realization allows you to be much more focused on what marketing and product need to do to improve the experience and thus reduce churn. If you can just get a customer to their first conversion, it will ultimately lift customer retention.

Stage 5: Establishing two metrics

Much of what I’ve discussed is focused on managing early-life churn, or, in other words, the number of customers who leave before they’ve really got value from your product. For most products, this juncture is where the customer is most likely to churn. However, to me, this situation doesn’t actually equal a churned customer. The person wasn’t really a customer, in my opinion. So, I would split your churn metrics into pre-Aha churn and post-Aha churn.

Effectively, this move is going back to having a base churn metric but adjusting it by removing new customers and only looking at the base churn of people who made it to Aha. This would be my recommended view of churn stats—two KPIs that give you a view of true product adoption (pre-Aha churn) and the proportion of previously happy customers who then decide to leave (post-Aha churn).

And now you are set up to improve your digital ad spend. Happy hunting, folks.