The ad industry continues to face seemingly endless challenges when it comes to accurately measuring the performance of digital campaigns. In an age where ad spend has been cut to account for economic uncertainty, it’s more important than ever to understand the pitfalls of the metrics you rely on, including and especially cost per acquisition (CPA).
By failing to grasp the shortcomings of their metrics, marketers are wasting ad dollars and spending valuable resources on improving a KPI that may actually tell them little about whether their ads have been effective. In other words, it is possible that improving your CPA won’t improve the quality—or efficacy—of your ad campaign.
CPA is a common KPI, but it is important to understand that not all CPA measurements are created equal. CPA can be calculated in many ways, and the results depend heavily on this choice. The techniques for measuring CPA are often referred to as attribution methods. The question an attribution method tries to answer is, “If someone converts, which campaign, strategy or tactic should get credit for that conversion, if any?”
The weakness of standard attribution methods is that they make little attempt to measure which advertisements actually influence the users, or to understand which conversions would have happened organically, without the ads.
The shortcomings of standard attribution methods
Single-touch attribution models (including first- and last-touch) assign all the credit to only one marketing touch point. One of the most common attribution models, last-touch, assigns all the credit to the last ad shown prior to the conversion.
While this often yields very palatable, very low CPA measurements, it can be manipulated and therefore misleading. Just because an advertisement receives last-touch attribution for a conversion doesn’t mean the advertisement caused the user to convert—for all you know, that person was already going to buy that product or visit that store before they even saw your ad. This is a subtle but critical distinction: Attribution and acquisition are not the same.
Multi-touch attribution models, while more complicated, suffer from the same fundamental shortcoming. Google released a study examining modern consumer journeys and found that the average journey now involves anywhere between 20 and more than 500 touch points. Multi-touch methods spread the attribution across these touch points, but the methods they use to do this are not scientifically rigorous with regard to understanding if and which advertisements actually caused users to convert.
If you remember one thing, remember that when you are using single- and multi-touch attribution, CPA does not actually measure your cost per acquisition. Instead, it measures the cost per attribution, and these attribution methods are not aligned with how effective your ads were at influencing users.
Incremental attribution isn’t easy, but it’s worth it
Incrementality is the more challenging, yet rewarding, attribution option. Skeptics of incrementality lament that it requires more cost and effort, but having an accurate and scientifically sound approach to measuring KPI is the bedrock of marketing efficiency and success.
Measuring CPA using an attribution methodology factors in how many conversions would have occurred without a campaign’s advertisements. The gold standard method for measuring incremental CPA is to conduct a randomized controlled trial (RCT), a rigorous approach to measuring the number of conversions a particular campaign actually drove. The Food and Drug Administration uses this scientific methods to measure the safety and efficacy of pharmaceutical products. Importantly, these trials remain accurate even if the campaign they are being used to measure is run alongside other campaigns, including other channels of media.
Typically, RCTs for measurement in advertising involve showing a public service announcement (PSA), or other irrelevant creative, to a group of users. While this comes at an expense, the reward is a truly accurate measure of how effectively campaign dollars are spent.
Beware of ghosts
The accuracy of an RCT depends heavily on how carefully the trial is executed. It is critical that users are randomly assigned to the ad group and the PSA group, and that they are treated identically except for the creative they see. Otherwise, the results will not be accurate—imagine if you put retargeted users in the ad group and random users in the PSA group. For this reason, it is important to understand the methodology being used to make incremental measurements.
Sometimes the term incremental CPA is used to refer to measurement based on attribution that uses ghost ads or ghost bids. These methods essentially try to simulate running a PSA control without actually spending the budget. These methods come with the risk that the measured performance completely depends on how accurately the PSA control is simulated. This can be a dangerous assumption, so unless you run a real control and find the ghost methodology produces accurate and consistent results, proceed with caution.
Looking ahead
In addition to incremental CPA providing a scientifically valid measure of campaign performance, the data generated by randomized control trials has added benefits. New statistical methods, borrowed from the field of precision medicine, can analyze RCT data to understand which audiences are most influenced by which creative. These insights can, in turn, be used to improve the performance of future campaigns.
While single-touch attribution and CPA have been used traditionally to help set KPIs for ad campaigns, it’s evident that the marketing industry needs to set a new bar for measurement and consider retiring legacy metrics that are hindering our performance. Many brands today believe they are facing performance challenges, when, in fact, most of their problems can be resolved with new measurement tactics.
While industrywide change is scary, it is crucial that we begin to approach measurement with a new set of eyes. Otherwise, not only do we risk getting left behind, we squander precious time, budgets and resources.
