License To Statisticize

You need a license to drive, a license to surgerize, a license to anesthesize…and it’s about time you needed a license to statisticize.

The abuse of statistics has been rampant in business circles for a long time, particularly as it applies to the confusion between correlation and causation. But the misapplication of statistical principles shows up in other, and sometimes subtle, ways.

***

A Centric Digital post titled How Banks Are Responding to the FinTech Craze reported on a study conducted by Statista. The headline on the Centric Digital post reads:

Most banks incubate to keep up with fintech.

This “conclusion” is based on Statista’s data which found that 43% of banks choose to participate in the fintech race by deploying startup programs to incubate fintech companies.

Source: Centric Digital

As you can see, the graphic says the percentages represent the “share of banks.” Exactly which banks are we talking about? The post states that:

“Banks around the world are feeling increased pressure to show they’re in step with the trend, though the approaches to this initiative are widely varied. The majority choose to outsource, either by initiating programs that incubate individual FinTech companies or by launching venture funds for them.”

Aha! So it’s a global sample of banks, is it? How many are we actually talking about here?

Well, don’t go to Quora for that answer. One payments/banking consultant adressing this question wrote:

“The US has slightly less than 8,000 banks/credit unions. For the rest of the world, you could estimate it by averaging 30 banks/credit unions per country, so for 220 countries/territories, that would be 6,600. So one could assume the figure would hover close to 14,600 +/- 1500 banks worldwide.”

Oh really? A simple trip to the World Council of Credit Unions site would have told this guy that there are 57,000 credit unions in 103 different countries.

Another Quora particpant, answering as Anonymous, said:

“Impossible to give an exact answer and the number would most likely be changing all the time. There are at least 1,500 individual retail banking corporations in the world, although the total number is likely to be significantly higher.”

S/he was smart to remain anonymous.

***

You’re probably thinking: Why don’t you just go to Statista’s site and find the data, Snarketing-boy? I did. It costs $49/month to get access to the data, and I wouldn’t pay 49 cents to get the answer. I did find out, however, that the sample was “global banks.”

My assumption here is that the data doesn’t apply to a global sample of the tens of thousands of banks out there. Instead, it probably applies to a small–in fact, likely a very small–sample of banks that are large enough to be global banks. You know, like 21 banks in total.

So when the chart says 43% of banks incubate fintech companies, what we’re possibly talking about here is 9 banks in total. Nine banks. You would never know that reading the Centric Digital post.

***

Here’s another abuse of statistics, a much more subtle one.

Let’s say there are ten factors (ATM/branch availability, fees, advice from family/friends, online/moblile tools, interest rates, and five other reasons labeled Reason #6 through Reason #10) influencing choice of banks, and that we survey five people, asking them to list the top three factors that influenced their decision, and that they respond as follows:

Respondent #       1          2          3          4          5
Reason #1          Rates      Rates      Fees       Fees       Rates
Reason #2          Tools      Reason #7  Reason #6  Reason #9  Tools
Reason #3          ATMs       ATMs       ATMs       ATMs       Reason #8

 

ATMs were listed as a reason by 80% of respondents, a higher percentage than any other given reason. ATMs must be the most important reason, then, right? After all, rates were only listed by 60% of respondents, and fees by just 40%.

But did the respondents list their reasons in order of importance, or just simply provide a list of factors they thought influenced their decision? If the former, then it would be hard to conclude that consumers “put ATMs first” when no one actually listed ATMs are their #1 influencer. If the latter, then it would be even more wrong to say that consumer “put ATMs first.”

This interpretation mistake happens so often, I could spend my whole week doing nothing but finding these things. Wait, what am I saying? This is what I spend my whole week doing.

***

A MarketWatch article titled Americans Put ATMs First When Picking a Bank reported that:

“The main factor for most people shopping for a bank isn’t personal recommendations, interest rates or online and mobile features–they want to make sure they have access to a branch and ATMs. [When] asked what the most important factors were when selecting a bank…for 41% of customers, [it] was ATM and branch availability.”

The study, conducted by Credio, doesn’t actually do what I illustrated above, however, since it asked consumers to list just one factor. There’s still an issue here, however.

Let’s say I go out and survey a respresentative sample of consumers and ask: “What color Ferrari do you like best?” and find that 98% like blue, 1% like red, and 1% like white. You might conclude that Ferrari should paint 98% of their cars blue. Only problem here is that the population of people who are actually going to buy a Ferrari is anything but representative of the overall population of consumers. And Ferrari buyers might all be in the 1% of the overall population who want red.

This is the problem with the Credio study: The people in the market for a new bank isn’t representative of the overall sample. Younger consumers are much more likely to be looking for a new bank, or switching than older consumers. It might politically-incorrect for me to say “Who cares what people over the age of 60 think?,” (face it–that’s what most Gen Yers think anyway), but for the purpose of trying to understand what drives choice of banks, that’s not a bad question.

When looking at respondents by age, the percentage of consumers between 18 and 29–the group most likely to be in the market for a new banking relationship–that listed ATM/branch availability as their primary factor is just 30%. OK, still the most frequently listed response, but nowhere near a majority.

***

Here’s another issue with MarketWatch’s conclusions from the Credio study. Among 18-29 year olds, 17% said “advice from family/friends” was the most important factor influencing their choice of banks.

But what exactly was the advice provided? It was likely advice about how good the referred bank’s rates, fees, mobile/online tools, or branch/ATM availability are. In effect, one of these four factors was really the influencing factor. The study confused influencing factors with sources of information. Bad move.

***

I’ve got one more issue with the conclusions from the Credio study.

When the data reported shows what just 25% of consumers list account fees as the most important factor, what does that imply? That three-quarters of consumers don’t care what the account fees are.

Good luck with that assumption, buddy.

Go ahead. Raise the fees on your checking account. It won’t matter as long you have convenient ATMs and branch availability, right?

Yeah, right.

***

Bottom line: The misuse and misinterpretation of statistics is out of control. With the explosion in the availability of data, every idiot (and that includes me) with a blog account is accessing that data and twisting it to suit whatever point they feel like making.

If you drive without a license, you can be fined. If you practice medicine without a license, you can be fined. You should be fined for misusing statistics without a license. The fines could pay down the US debt.

Ron ShevlinRon Shevlin is Director of Research at Cornerstone Advisors. Check out more of his ideas and research on Cornerstone's Insight Vault. And don't forget to follow him on Twitter at @rshevlin.

This article was originally published on June 15, 2015. All content © 2018 by The Financial Brand and may not be reproduced by any means without permission.

Comments

  1. A majority of the people surveyed at my house enjoyed this article.

Speak Your Mind

*

Show Comments