Big Data: Can’t Get There From Here

Subscribe Now!

Stay on top of all the latest news and trends in banking industry.


Have you seen how much money professional baseball players make? Millions. And I wan’t me some of that, so I’ve decided to become a professional baseball player.

Oh sure, there are a few things I have to work on, like my hitting, throwing, catching, running, and inability to stay up past 10:30pm.

But every time I read about how much these guys make playing baseball, I know my efforts will be worth it.

Now, I know what you’re thinking. You think I can’t do it. You think that I — as they say in Maine — “can’t get there from here.”

And, of course, you’re 100% correct. But my self-delusion is no different than the delusion held by many marketers who think that somehow, magically, Big Data will transform their companies into industry leaders.

Never mind, of course, that no one is willing to agree on exactly what Big Data is or isn’t. Consider this comment someone left on a blog:

“Publishers have no insight into the effectiveness of their marketing. We firmly believe the answer to this problem lies in “big data” – that publishing can dramatically improve its return on marketing investment through the methodical analysis of campaigns, in combination with more agile marketing techniques.”

Apparently, Big Data = Analytics. Who [email protected]#!

This is hardly an isolated example.

Personally, I think of Big Data as data (and data sources) that is created by human behaviors and conversations that occur online, as well as data that is created because so many devices (not just PCs, tablets, and cellphones, but cars, refrigerators, and whatever other mechanical devices are connected to the Internet and can become a source of data).

But I’m not here to impose my definition of Big Data on anyone. I don’t care how you define it. I care about how you’re going to use it in a way that improves the effectiveness, efficiency, and profitability of your company. And, in that context, I think that there are (at least) three hurdles that few Big Data advocates recognize:

1. More data will cause more problems, not less. The ability to capture people’s online behavior adds a lot of data to the marketing arsenal. And there is a lot of discussion about how “unstructured” data — e.g., conversational data — that occurs in social media channels can be captured and analyzed. But unstructured or not, the act of capturing data requires us to label and categorize it — that is, to define it. The act of defining data opens the door for differences and errors in interpretation.

Example: Assume that a scanner can sense when a customer goes into a store, and record it as a “store visit.”

Wouldn’t it be great for retailers to know how many of their customers visited a store,when, and for how long? Reconciling with a transaction system can help determine if a sale was made during a particular visit, and if not, perhaps signal the retailer to send a marketing message to the effect “Couldn’t find what you were looking for? Maybe we can help!”

But what if that store was in a mall. And what if my reason for being in the store was simply that the parking spot I found was close to the store. Oh sure, maybe your “interaction system” (in contrast to your “transaction system”) will disregard short visits. But right off the bat, we’ve introduced two decisions that have to be made about our new “big data”: What exactly is a “store visit” and “how long does a visit have to be to be considered a store visit.”

Now multiply these decisions by the multitudes of data sources and data types that Big Data advocates rant and rave about.  I think we’ve exceeded humans’ data processing capacity.

Aha, but no worry, the Big Data advocates say, that’s what the technology is for. Which takes me to the second big Big Data hurdle…

2. Data won’t analyze itself. I’ve worked for a database marketing company. You know who does the analysis? Really smart people with PhDs in statistics.

Your company likely has one, maybe two problems. Number one, you might not have a lot of smart people. That’s your subjective call (I have my opinion, but sharing it will only make people think I’m cranky and elitist). The more likely case is that your firm doesn’t have a lot of Statistics PhDs on staff.

The Big Data advocates know this, and assure use that our schools will start turning out kids whose knowledge of statistics is far greater than it is today. Okey dokey.

But let me share a secret about today’s Statistics PhDs that they won’t be happy that I’m sharing: They don’t know how to incorporate new data sources. Let me restate this in a more positive way: They’re still learning how to incorporate new data sources.

The purchase propensity models that they’ve developed use the data that’s available today, which, I guess for lack of a better term, could be considered Little Data. Little Data is predominantly demographic and purchase (i.e., transaction) data, not behavioral and conversational data.

Now let me explain something that Big Data advocates don’t seem to understand: When you’ve developed and tested a propensity model in the real world, making changes to that model (because you have some “new” data) is a risky, and potentially expensive endeavor.

Database marketers like to test new data sources before making big investments in the capture, storing, and utilization of that data for marketing purposes.

How is all this going to happen in the Big Data advocates’ vision of Big Data Nirvana? Who’s going to do all of this?

No worries, say the Big Data advocates. Companies will start hiring more Data Scientists (the more adventurous of which will call themselves “Data Whisperers” causing me to barf my lunch).  If you believe this, you need to read on the third hurdle…

3. The cultural hurdles will be tough to overcome. How many Data Scientists do you think the marketing department of a large retailer or financial services firm is going to hire? Database marketing can’t get the budget to increase its staff today, how is this magically going to change in the near future? Oh, these Data Scientists are going to reside in IT, which charges back its expenses to the rest of the organization, and is under constant cost pressure?

The budget issue isn’t even the biggest of the cultural hurdles.

Here’s the problem, and nobody wants to admit it: The CEO or CFO can look at a TV ad and say “I love it” or “I hate it.” They can’t look at a propensity model and say “I’m not crazy about that model.”

Organizations are typically run by people with years of experience in that industry and/or that firm. They develop beliefs about what works and what doesn’t. Ceding decision making power to the “data” or the “analytics” requires a leap of faith that is typically only found in firms on burning platforms, or in Brad Pitt movies.

Now don’t get me wrong — if Hollywood wants to turn this blog post into a movie, I’d be more than happy to have Brad Pitt play me in the movie.

But in the Big Data advocates’ view of the world, their envisioned future is fantasy, not science fiction.

You Can’t Get There From Here

There are some major hurdles in the way Big Data having the impact that advocates envision it to have. And, I’m not very confident that many companies — as they say in Maine — can get there from here.

At least not in the short-term. Because I do believe that the ability to capture and analyze behavioral and conversational data will transform marketing. But I think it’s going to take 25 to 30 years before we really see marketing departments with strong competencies in Big Data analysis. OK, maybe 20. 

And I really don’t care if I’m off by a few years or not, because I better be retired in Maine by then. And no, you can’t come visit me — because you can’t get there from here.

This article was originally published on . All content © 2022 by The Financial Brand and may not be reproduced by any means without permission.