This is a continuation of a previous article called Calculating Digital Signage ROI: The Ground Rules.
In this article, we're going to focus on some things NOT to do while
performing an analysis of your digital signage data.
Understand the Limits of Your Data Set
We've
all heard the saying "numbers don't lie," and that's true.
However, it's unfortunately also true that people do lie.
Methodically. And perpetually. And in fact, even the
best-intentioned people can get into trouble when analyzing data
because their own hopes and beliefs will alter the way they interpret
their numbers. So I'd like to point out a couple of common
pitfalls and misunderstandings about data analysis that will hopefully
spare you the embarrassment of being called a cheat and a liar the next
time your customers ask you for the hard numbers.
First, correlation DOES NOT imply causation.
In
other words, just because two events happen to occur at about the same
time, you can't automatically assume that one of them caused the other
-- or will continue to do so in the future. Without a control
case, you'll never be able to accurately measure the impact of your digital signage network.
You might like to think that the 30 second spots that you're running
for cotton tube socks were responsible for the 15% up-tick in tube sock
sales, but in reality, other factors -- perhaps unusually cold winter
or a new fashion fad that idolizes 1970s basketball stars -- could have
caused the surge as well. The only way to be sure is to measure
your signage-enhanced numbers against numbers from a known control
group, for example a store with similar demographics and geography, but
no signage network. If I've confused you with my explanation of
this concept, you might want to check out this link, which gives a brief introduction to some common statistical notions.
Next, work blind when you can.
As
I said before, even the most honest people can be influenced by their
own subconscious desires. That's why any serious scientific
analysis takes place under "blind" or "double blind" conditions, where
researchers are not aware of whether they're analyzing their target
dataset or that of a control group. Digital signage analysts can
work much the same way by simply removing any identifiers from their
data set and having more than one person analyze the data. If you
can afford it, you might want to consider having an independent auditor
come in to check your numbers. The bigwigs like ACNielsen and Arbitron
have all sorts of clever ways for measuring traffic in your locations,
and years of experience in determining the effectiveness of marketing
campaigns in retail locations. Also, paid auditors can lend an
air of legitimacy to your data, and they have (slightly) less incentive
to cook the books (Arthur Anderson consultants notwithstanding, of
course).
And finally, don't affirm the consequent
When
people say something like, "you can't prove a negative," what they're
probably talking about is the 3,000 year-old proof that you cannot prove an argument true by affirming it's consequent. Consider the example:
[ 1 ] If the digital signage network is effective then we will see an increase in sales of promoted items.
[ 2 ] Sales of promoted items have increased.
[ 3 ] Therefore, the digital signage network is increasing sales.
These
are perfectly logical-sounding statements, and a great way to "prove"
the same kinds of things that I mention in the "correlation does not
imply causation" example above. The problem is, you actually can't
prove this argument. Even though the premises might all be true,
the conclusion is not necessarily implicated by the first two clauses.
I
think this is a good place to leave off for now. In a future
article, I'll start looking at different ways to model ROI, as well as
some techniques for getting the data you need to tweak your network's
performance.