Do you remember the joke about a drunk man crawling around on his hands and knees under a street light late at night looking for his wallet? A passerby asks the man if he is sure this is where he dropped the wallet, to which the drunk replies, “I actually think I dropped it across the street.”
“Then why are you looking over here?” the baffled passerby asks.
“Because the light is better here,” explains the drunk.
Unfortunately, the same can be true for the “key performance indicators” (KPIs) that financial marketers often rely on. Instead of matching KPIs to the ultimate desired objectives, many times we find ourselves focusing on “where the light is better” — i.e. what is easiest to quantify.
This happens for a variety of reasons:
Lack of timely data – This can be due to either short decision cycles (e.g. campaign to campaign) or long measurement periods (e.g. 5-year retention).
Technological constraints – The desired data may not be capable of capture (e.g. referring web pages without links).
Indirect purchase paths – Customers will likely see multiple messages across various channels before taking the desired action.
Privacy and legal constraints – Besides governmental regulations, your own website or partner’s privacy policies may prevent data use.
Poor data quality – This can be due to human data entry errors or lack of input validation.
Data capture constraints – These can include a lack of control of the data capture process (e.g. a separate line of business) or the nature of the process (e.g. multiple response channels which don’t all have ability to capture data).
High data acquisition costs – The cost to purchase the data or the investment required to capture the data might be too high.
Third-party gatekeepers – The data may be owned/controlled by a third party not willing to give up control of the data.
High opportunity costs – Holding out a large control group might cost too much in lost sales.
Lack of common ID – There may be no effective way to tie together disparate pieces of data (e.g. linking online with offline behavior).
In the above situations where we cannot track the desired information directly, we instead tend to go where “the light is better” — a.k.a. “proxy KPIs.”
Proxies — The Good, Bad and the Ugly
A proxy KPI can provide us with a benchmark to rate our performance. The problem with this approach is that the proxy soon becomes the new goal, usurping the original objective.
For example, suppose your goal is to acquire new checking households who will generate a 25% ROI over their lifetime. If the average retention rate for new households is three years, then you would need to wait that long after an acquisition campaign completes to determine if it was effective.
Instead, you might use a profitability model based on the first three months of behavior. If you are running campaigns quarterly or more often, then even three months is too long. You might default to looking at incremental cost per account instead to enable you to tweak your campaigns.
You may target only certain households to increase the likelihood of profitability on the back end, but that ignores the effects of your marketing efforts. Different incentives will not only attract different responders, but will also motivate different behaviors — making it more difficult to predict their lifetime profitability. Regardless of your original objectives, your marketing decisions soon become based on short-term, readable proxies, thereby becoming your new objectives.
This can be true for any of your marketing objectives. For example, if you are acquiring new home equity lines through digital display, your goal might be cost per new account, while your proxy KPI is leads per click.
Because of the difficulty in obtaining the right information, we settle for proxies and mistakenly change our objectives because we cannot track them directly.
What is the Solution?
There are a few options to improve your ability to measure your marketing success:
1. Fix your data capture problems – permanently or temporarily. While not all of the problems can be fixed, there are workarounds to many of them. Consider a temporary solution whereby you capture the more accurate data (i.e. tied more closely to the original objective) on a less frequent basis. This could solve issues to the following:
- High data acquisition costs (not having to purchase the data that frequently)
- Third-party gatekeepers / privacy and legal constraints (have them provide anonymous analysis)
- High opportunity costs (only use large control groups sparingly)
To solve the poor data quality and data capture constraints, start viewing them as customer service issues instead. This can help raise the priority to fix the data capture problems.
For indirect purchase paths, the optimal approach is to use fractional attribution, which takes into account multiple marketing messages.
2. Develop a compromise objective that can be tracked. Try breaking up the larger goal into a series of more manageable ones. For example, if your current objective is “acquire new, highly-profitable customers via display advertising,” it assumes that you are acquiring the right prospect AND they will behave a certain way as a customer. The latter will easily depend on your customer onboarding, cross-sell and retention activities, which have little to do with the acquisition.
Instead, set your objective to acquiring the right types of customers that have potential to become very profitable. This can be accomplished through a combination of targeting and offer design to help self-select the best future customers.
3. Embrace proxies with care. Select a single KPI that can be closely tied back to the original objective. While multiple KPIs are useful to understand a marketing effort, it is extremely difficult to optimize something with multiple goals. Combining the proxies into a single KPI simplifies the effort. For example, if you are concerned about response rates and deposit balance, measure balance per unit contacted.
After selecting a proxy, verify it is indicative to the true objective by analyzing similar customers to determine the correlation.
Even in the best of circumstances with properly designed tests, no measurement is perfect. Results can vary for a number of reasons, which means selecting an acceptable KPI range instead of a specific target will help ensure you are making decisions that will stand up to all that we cannot control.
And remember, if you want to find the right answers, look in the right places… not just where the light is better.