A couple years ago many of you were stunned to learn that the pollsters and election forecasters called the presidential election wrong. But, party affiliations aside, all marketers can learn an important lesson from the last election: be careful not to put too much faith in data as absolute predictors of future outcomes.
“The uncertainty we have to account for is the uncertainty of things we don’t know we don’t know,” said Dr. Pradeep Mutalik, a research scientist at the Yale Center for Medical Informatics, who says the results of that presidential election made a mockery of analytical election forecast modelers.
Big data and predictive marketing have changed how companies think about marketing. We’re supposed to know what will happen within a decimal place of precision when we plan the next six months. Suppose your CEO asks you: “If I give you [n] dollars for marketing, how many new customers and dollars of revenue can I expect?” You dutifully grind through the data and produce a report that declares some expected number of customers and revenue. And there it is: the defining moment when you and the executive team look one another in the eye and agree that this is how you will be measured month over month, quarter over quarter.
Now ask yourself: has everyone accounted for the uncertainty of the things you don’t know you don’t know? Unlikely. Did your model account for economic uncertainty? Or seasonality? New competitors? How about turnover on your team? And what about your friends in the sales organization – think anything might change that would impact the outcome?
The box you’re in isn’t the fault of the model, the data, or the software. No, it’s in the assumption that you Mr./Ms. Marketer have the immutable power to predict outcomes to a level of precision and accuracy that is not achievable.
You must shift your thinking to look at data more like clues than explicit instructions to guide your decisions. Just because a marketing campaign last quarter generated 128 new names doesn’t guarantee you’ll get the same results if you run the same program again next quarter. But if you compare that program against fifty campaigns you ran last quarter, you could stack rank them to determine the top performing campaigns for adding new names. The top three campaigns may reveal some clues as to why they were more successful than the others – then again, they may not.
Here’s advice on how to live with big uncertainty in your marketing analytics and forecasting:
- Consider how much data and what period of time are you looking at. Are you confident there’s enough “there” there to stand on?
- Know exactly how the analytics are derived. If you don’t, you risk making decisions on misinterpreted data and exposing yourself to a loss of credibility. For example, before you show data on multi-touch influence to sales pipeline, reverse engineer a handful of deals to understand how programs and people are assigned credit.
- Temper your faith in predictive. Just because a model predicts an outcome doesn’t mean it’s going to happen. Models can be useful as guides, but no marketing model can fully account for the vast number of variables in play.
- Planning based on what you already did discounts the untried. When past performance is the only input to planning you miss out on the potential of new ideas.
- Your past performance is a better predictor of future outcomes than someone else’s. Sure, benchmarks are interesting points of comparison but the farther they stray from your specific experience the less reliable they are.
None of these will help you unless you take seriously the need to get everyone who consumes marketing analytics on the same page. This isn’t as simple as it sounds. On any given day, you and the marketing team should be prepared to talk about your plans and results in meaningful, relevant ways that anyone at your company with a vested interest in marketing (who doesn’t?) can understand. That’s right, stakeholders from the board on down have to buy into your approach, your data, your systems and most of all – you. This is a strategic endeavor that, if ignored, can leave you exposed to anyone who wants to use data to drive their agenda. One of the biggest challenges to winning hearts and minds is this: not everyone “gets” marketing, especially modern marketing analytics. Their ideas about marketing may be outdated, overly complex, unrealistic, or biased.
Reflecting on that infamous presidential election, Dr. Mutalik points to innumeracy – a lack of basic skills in math – as a factor that played into this past election. He notes that a large portion of the population “can be led astray by statistical statements and quantitative arguments in news stories.” Is a lack of basic skills in marketing analytics at your company a risk factor for your marketing team? Can decision makers be led astray by data that’s sourced and interpreted by people at your company from other departments? Absolutely. I’ve seen good analytics from marketing disparaged just because they came from the marketing automation platform. I’ve seen executives and board members request reports that won’t tell you anything actionable. I’ve seen marketers give-in to, or green-light reports they know are wrong, or misleading.
What’s lost in all of this is a simple idea: the main use of marketing data and analytics is to continuously guide marketers and their companies toward better business outcomes. It’s about understanding marketing vs. marketing, not marketing vs. sales; in other words, you want to optimize marketing ROI, not whether it’s better to spend a dollar on sales or marketing. If you keep this simple idea at the forefront of everything marketing, your chances of putting data to work for your business are far better.
True high precision forecasting is still a ways off for marketing. Until then, account for the uncertainty of things you don’t know you don’t know.