The Market Research Hall Of Shame

MSN Money recently published its Customer Service Hall of Shame, which found banks like Bank of America, Citibank, and Wells Fargo in the “Bottom 10”.

As a way to generate publicity, this is an infallible approach. Survey some consumers, ask them to rate a bunch of big companies on customer service, and blare the results all over the press and the Internet heralding the worst customer service providers in the country. Great job, MSN Money!

But as serious market research, this is garbage. How did MSN Money come up with its list? By first asking consumers to nominate firms to the potential list of bad service providers. So right off the bat, when asking consumers to rate those firms in the follow up survey, two things are guaranteed:

  • Only large firms will make the list, since the sample size for smaller or regionally-focused firms won’t reach critical mass. Is it true that only large firms deliver bad service? No.
  • More importantly, there’s no way to determine the difference in ratings between the “worst” firms (those included in the survey) and the “best” firms. Who those firms are, of course, is anybody’s guess since they weren’t nominated or included in the survey.

How can anybody evaluate the context of the results?

One of the firms on the dishonorable mention list is Macy’s. Yet only 10% of the customers surveyed rated its customer service as “poor” and, in fact, 60% rated it “good” or “excellent”! So tell us, MSN Money — how does this compare to the very best customer service providers? You don’t know. [By the way, of the six firms on the dishonorable mention list, five of them were rated as “good” or “excellent” by more than half of their customers].

What this is is nothing more than a shameless hatchet piece designed to embarrass a bunch of big companies and generate publicity for the firm releasing the survey. If this happens to your firm, you should:

1) Publicly ignore the study. By issuing public responses to the study, firms like BofA and Comcast wrongly bestow legitimacy to it. And with all due respect to BofA and its customer service executive (who the response came out from), the BofA response was, in my opinion, weak and confusing. While it would be my inclination to blast the methodology of the study (as I’m doing here), a wise firm would simply ignore it.

2) Privately strike back. If I were the top market research exec at any of the firms on the list, I would immediately send out a memo to my market research providers letting them know that if any of them participated in any of these “designed-to-be-a-hatchet-job” studies, they would be banned from bidding on any jobs in my firm for three years.

The firms on the list represent 16 of the largest firms in the country, and presumably, all of them spend a fair penny on market research. Do you think Zogby would have been so quick to take this job if it thought it would be shut out from working with these firms — and who knows how many others? I’m not saying that market research firms should clear studies with other clients. But you couldn’t convince me in a million years that no one at Zogby knew how the results of this study were going to be used.

So congratulations MSN Money — you’re the first inductee in my Market Research Hall of Shame.

Technorati Tags: Marketing, Market research, MSN Money, Customer Hall of Shame

[Personal note to John Dawson: Is there someone you’d like to nominate?]

This article was originally published on . All content © 2019 by The Financial Brand and may not be reproduced by any means without permission.