There’s something that’s bugged me for years, and I’m coming to the conclusion that — given its widespread adoption — I’m the only person on the planet who feels this way.
It’s the use of the phrases “more likely” and “more ___ than” when analyzing research statistics or performance metrics.
Example: VentureBeat ran the headline “Facebook mobile ads are 4 times more engaging than Twitter ads.” According to some study, Facebook’s mobile ad click-through rate is about 1.1 percent, more than four times higher than Twitter’s 0.266 mobile CTR.
Four times as many mobile Facebook ads were clicked on than mobile Twitter ads, but that does not necessarily mean that Facebook ads are “more engaging” than Twitter ads.
To make that claim, you would need to have a measure of “engageability.”
So, let’s say that graphic appeal, legibility, use of white space, and screen placement all contributed to “engageability.” If you measured a Facebook ad and a Twitter ad and found that the FB ad scored an 80, and the Twitter ad a 20, then you could say the Facebook ad was “four times more engaging.”
Simply taking the actual CTRs — 1.1% and .266% — does not prove that Facebook ads are more engaging.
Maybe the higher CTR for FB ads is due to the demographics of users. Maybe it’s because the ads that run on FB are ads for different types of products than those advertised on Twitter. Maybe marketers are running their Twitter ads at the wrong time of day.
This misuse of the “more likely than” phrase shows up in other research analyses, as well. If 50% of Gen Yers and 25% of Boomers use public transportation, Gen Yers are not “twice as likely” to use public transportation. In fact, you can’t even say that “twice as many” Gen Yers as Boomers use public transportation.
[Oh, c’mon. Seriously? I have to prove that to you? OK, If there are 1 million Gen Yers, and 50% take public transportation, then 500k Yers use public transportation. If there are 10 million Boomers, and 25% take public transportation, then 2.5m Boomers use public transportation. Despite the differences in percentages, five times as many Boomers use public transportation].
Bottom line: If elected president in November, I will make it illegal to misuse the phrase “more likely than” when reporting research studies.