Cause and Effect

Cause and Effect in Marketing

Correlation versus Causation when looking at your marketing data.

Daniel Whittington: Welcome to the American Small Business Institute. I’m here with Stephen Semple. And today we’re going to talk about data and analytics as it relates to cause and effect. Or as Stephen put it-

Stephen Semple: Correlation versus causation.

Daniel Whittington: Yes, okay. Post hoc ergo propter hoc.

Stephen Semple: Yeah, something along that lines.

Daniel Whittington: So what’s our story?

Stephen Semple: We’ve been doing a lot of videos on putting together dashboards and collecting data, and things along that line, which is all excellent. We want to have data. We want to know what’s going on in our business, but one of the most common problems I see is people will mistake correlation with causation.
So in other words, they’ll see two things happen, or they’ll see one thing happen, “Boy, I changed this, and then this happened, so, therefore, this was the cause.” And it may not be. It may not be the cause. It may just be those two things are happening at the same time. It may be that there’s a third element, outside of your business, that’s actually creating the cause.

Daniel Whittington: Like someone on the phone started asking a different question all the time, and now people are actually going to the website.

Stephen Semple: Right. One of my favorite ones is, let’s picture you’re in the wine industry, and you’re selling Merlot-

Daniel Whittington: Oh, no.

Stephen Semple: Or you’re selling pinot noir.

Daniel Whittington: I know where this is going.

Stephen Semple: Right. And you can look this one up on the internet. The Merlot effect. The movie Sideways comes out. A guy does a passionate speech about why Merlot sucks, and pinot noir is great, and guess what happens. Pinot sales go up.
No, if you had launched a brand new campaign at that time, you’d be going, “Woo hoo!”

Daniel Whittington: “We did it!”

Stephen Semple: “We did it!”

Daniel Whittington: And if you haven’t watched the movie, which a lot of people hadn’t because it was not a blockbuster-

Stephen Semple: It wasn’t, but it’s been well documented, because companies that did know advertising around that time, had a lift. It’s been really well documented that that affected the entire industry. So if you had created a brand new campaign, you go, “We found the secret sauce. This campaign is killer.” Meanwhile, no.

Daniel Whittington: So how do you know? Because the whole point of analytics in my brain was to tell you what happened, right? That’s the whole point. To tell me what happened.

Stephen Semple: Right.

Daniel Whittington: So how do I know what to highlight and what not highlight?

Stephen Semple: Excellent. Let’s go old school. If we go back to the days of direct mail, one of the things that all really great direct mail marketers did was run what’s called a control.

Daniel Whittington: Okay.

Stephen Semple: So they would have an ad that they’re doing, and when they’re doing a new ad, they would run those two ads in parallel. So they would run a little bit of the control, and they’d run a little bit of whatever the new direct response campaign is, and look at the comparative result between the two of them because if something like the Pinot Effect came along, you would get a lift in both campaigns. And you go, “Holy crap. It’s not the campaign, it’s something else going on. I don’t know what it is, but there’s something else going on.”
Just like if you were running a Merlot campaign, and you got an ass-kicking, you would get it on both of them.

Daniel Whittington: Right.

Stephen Semple: And really, the question is, what is a comparison between the two, because then that would actually tell you which one’s the better campaign.

Daniel Whittington: So what if you’re doing things like, sending people to your home page?

Stephen Semple: Right.

Daniel Whittington: How would you do a comparison on that, to understand that this bump from traffic right after you launched this thing really was because of the marketing that you did.

Stephen Semple: I could run two homepages. I could have my existing homepage, and I could run a test on the new homepage, and drive traffic to both.

Daniel Whittington: Okay.

Stephen Semple: And see which one-

Daniel Whittington: Which is a slight, different little URL tag, or something?

Stephen Semple: Right, yeah. So then my existing homepage is the control. I know how that’s working. I can compare it to the new homepage. I can run, when I’m running AdWords, I should always keep a couple of the AdWords from an old campaign running, even if it’s just a little bit of it because then it gives me an idea, “Is there something else going on that creates a lift?”

Daniel Whittington: So it seems like-

Stephen Semple: This happens all the time.

Daniel Whittington: It seems like what you are suggesting is that, to use data wisely, there always needs to be a Venn diagram of campaigns.

Stephen Semple: Yes.

Daniel Whittington: No matter what you’re doing, whether it’s web building, AdWords, direct mail, anything, they just need to always be a Venn diagram of old and new?

Stephen Semple: Yeah.

Daniel Whittington: Side by side.

Stephen Semple: Absolutely, if you can. Now, I’m also practical. I know sometimes the size of a campaign you run, you can’t run a control and the other, which then just means you always have to be a little bit suspect.

Daniel Whittington: Little skeptical.

Stephen Semple: And don’t always immediately go to, “Oh, this caused this.” What I’ve found is if you’re always challenging that, often you can come up with the right answer. Because sometimes we just have to because it’s too small of a campaign.

Daniel Whittington: So that means part two is, always remember what you need out of this, and be aware that you might find it in it, because you need it so badly.

Stephen Semple: Right.

Daniel Whittington: I used to work a lot with digital marketing people, working on stuff I was doing, other things with, and they would generate reports, and there would be 50 possible reports, and they would generate the three that they were looking for.

Stephen Semple: Right.

Daniel Whittington: Right?

Stephen Semple: Yeah, absolutely.

Daniel Whittington: And then present it in a meeting, and I would go, “Huh.” And I would go back and generate another 30 that said totally opposite things from the same information. Remember that book called, How To Lie With Statistics?

Stephen Semple: Yeah, well there are three types of lies. Lies, damn lies, and statistics, right?

Daniel Whittington: So the two principles, I’m feeling is, to be careful what preconceptions you’re bringing to the table because you’ll often find them, and two, a Venn diagram overlap of old and new could help you weed out X-factors.

Stephen Semple: And just recognize these outside factors are way more common than you would think. Another great example is the Ford Bronco. You know the two-door Ford Bronco?

Daniel Whittington: Mm-hmm (affirmative).

Stephen Semple: It was going to be discontinued, and then O.J. Simpson and his buddy decide to go on a big, long car ride on the highway. That actually lifted sales enough that they extended the Ford Bronco for another couple of years. And then he got convicted, and the Ford Bronco sales went boom, and then discontinued.

Daniel Whittington: All right, well thank you, sir. Appreciate it.

Stephen Semple: All right, thanks, man.

Daniel Whittington: We’ll see you guys next week.

Scroll to Top