Why Marketers Should Examine Vendor Data Quality

Many inaccuracies lie under the hood

An interview with:
Allen Stern
Director of Programmatic Media
AKQA

Facebook’s ongoing scandal with Cambridge Analytica is forcing people to think more critically about data brokers. Allen Stern, director of programmatic media at ad agency AKQA, spoke with eMarketer’s Ross Benes about why advertisers should investigate the quality of the data they obtain from vendors.

eMarketer:

When you’re selecting a data vendor, what do you look for?

Allen Stern:

What we’ve tried to do as an agency is validate some of those data sources. If we’re buying data on men ages 35 to 54, are we really getting data on men 35 to 54? Rather than just taking it for granted, we’ve discovered that it is really worth it to try and validate that data.

eMarketer:

How do you do that?

Allen Stern:

We use surveys. For example, we serve an ad to the [target audience] and then follow up with a survey to see what kind of responses we get.

eMarketer:

What types of things are you asking about in the surveys?

Allen Stern:

We started with demographics—easy demographics like gender and age. Then we went to the next round, with household income and whether there are children present in the household. The phase we’re in right now is purchase data. We’ve done the surveys in phases since last year. We try to test something new each time.

eMarketer:

Once you have that data, what do you do with it?

Allen Stern:

We compile it and talk to our client about the results. When I can go into a business pitch and say, "Hey, client, guess what—we understand 20 different data vendors on gender, and if you want to reach men ages 18 to 34, we know the right ones to use," that's compelling.

eMarketer:

Isn’t it expensive to do this type of research?

Allen Stern:

There is a cost to it, but it’s not crazy expensive. It definitely takes resources to do it. It’s portions of people working on it part of the time. It’s a cost, but it’s part of our value proposition.

A willingness to be transparent is an implied endorsement of your own data.

eMarketer:

Have you found anything surprising during your tests?

Allen Stern:

What is surprising is the wild inaccuracy of some vendors. Like, if you didn’t buy any data at all, you would reach more of your target for less money than if you bought the data. That sort of rocked us.

eMarketer:

Were those wild inaccuracies outliers?

Allen Stern:

Yes, but they are big names in the data field. So many folks model their data, but how good is that modeling? That’s what it really calls into question. If the [sample size] is 10,000 people and the model is 100 million people, that’s quite an extrapolation, isn’t it?

eMarketer:

Do you get overwhelmed by how many firms are selling data?

Allen Stern:

There are a lot of folks trying to make a lot of money in this business, and data is hot right now. You're going to find people trying to do what they can. We see so many vendors coming across with data, and we really grill them. And we go with hardly any of them, but there's a lot that we come across. It's a resource demand to try to filter through it. And my biggest fear is that I'm going to miss something that's terrific.

eMarketer:

When you're shopping around, how are data vendors able to stand out from each other?

Allen Stern:

A willingness to be transparent is an implied endorsement of your own data.

Interview conducted on April 30, 2018

"Behind the Numbers" Podcast