Mixpanel published a widely covered blog post, titled BS Metrics. According to the post:
“Sadly, we haven’t moved forward over the past decade despite our whole industry becoming smarter about how it measures and analyzes data. Companies still pitch investors with a cumulative user sign up graph, sell advertisers on how many pageviews they get, and bamboozle reporters with the biggest numbers they can find regardless of whether they correlate to success. Companies need to start using a new set of metrics that don’t simply make them feel good. They should use actionable metrics that provide insight, provide guidance, and help businesses make better decisions.”
My take: Good points, can’t argue with that. But BS metrics can still serve a useful purpose. (And if investors, reporters, and advertisers don’t demand better metrics, than don’t blame companies for reporting BS metrics!)
Why do you measure what you measure?
“To know how we did, how we’re doing, and predict how we will do,” you’re likely to say. A great answer. To evaluate and track performance is an excellent objective for a metric.
But “actionable” metrics may have negative side effects.
Revenue per team may be a good, actionable indicator of the performance of the various teams that exist at your company.
But what if one client buys from multiple teams? Let’s say I do such a great job with a client (hey, it could happen) that, as a result of my efforts, the client buys products/services from another team at my company. Their revenue goes up, mine doesn’t, they get credit, and I get screwed.
Eff that metric.
Bottom line: Different metrics can serve different purposes. Two in particular (and two that serve in defense of BS metrics) are motivation and alignment.
A metric like pageviews may be BS, may not be actionable, and may have no direct connection to bottom line performance, but: 1) If there’s no (or very little) incremental cost to measure it, and 2) It helps get people on the same page (pun intended), then there may be benefits to tracking the metric.
Companies should approach the identification of their metrics as if they were constructing a portfolio.
If you were constructing an investment portfolio, you would likely include investment vehicles with different risk profiles and the such.
Same thing with performance metrics. There is no reason why all your metrics need to be “actionable” metrics.
The author of the Mixpanel blog post writes:
“My experience has shown that companies should start by tracking a single actionable metric that they can literally bet the company on. I call this their One Key Metric (OKM). The best part of OKM is that companies can measure other things related to it, to understand how to improve it. Understanding your OKM often leads to deeper, more valuable questions.”
I’m inclined to agree. The challenge, however, is in defining the right OKM. It’s easier said than done.
But OKM doesn’t mean tracking just one metric. And those other metrics can be BS metrics, as long as you and management know they’re BS.