Calculating Digital Signage ROI: Understanding the Limits of Your Data
Author: Bill Gerba on 2005-03-30 12:02:13
This is a continuation of a previous article called
Calculating Digital Signage ROI: The Ground Rules.
In this article, we're going to focus on some things NOT to do while
performing an analysis of your digital signage data.
Understand the Limits of Your Data SetWe've
all heard the saying "numbers don't lie," and that's true.
However, it's unfortunately also true that people do lie.
Methodically. And perpetually. And in fact, even the
best-intentioned people can get into trouble when analyzing data
because their own hopes and beliefs will alter the way they interpret
their numbers. So I'd like to point out a couple of common
pitfalls and misunderstandings about data analysis that will hopefully
spare you the embarrassment of being called a cheat and a liar the next
time your customers ask you for the hard numbers.
First,
correlation DOES NOT imply causation.
In
other words, just because two events happen to occur at about the same
time, you can't automatically assume that one of them caused the other
-- or will continue to do so in the future. Without a control
case, you'll never be able to accurately measure the impact of your
digital signage network.
You might like to think that the 30 second spots that you're running
for cotton tube socks were responsible for the 15% up-tick in tube sock
sales, but in reality, other factors -- perhaps unusually cold winter
or a new fashion fad that idolizes 1970s basketball stars -- could have
caused the surge as well. The only way to be sure is to measure
your signage-enhanced numbers against numbers from a known control
group, for example a store with similar demographics and geography, but
no signage network. If I've confused you with my explanation of
this concept, you might want to check out
this link, which gives a brief introduction to some common statistical notions.
Next,
work blind when you can.
As
I said before, even the most honest people can be influenced by their
own subconscious desires. That's why any serious scientific
analysis takes place under "blind" or "double blind" conditions, where
researchers are not aware of whether they're analyzing their target
dataset or that of a control group. Digital signage analysts can
work much the same way by simply removing any identifiers from their
data set and having more than one person analyze the data. If you
can afford it, you might want to consider having an independent auditor
come in to check your numbers. The bigwigs like
ACNielsen and
Arbitron
have all sorts of clever ways for measuring traffic in your locations,
and years of experience in determining the effectiveness of marketing
campaigns in retail locations. Also, paid auditors can lend an
air of legitimacy to your data, and they have (slightly) less incentive
to cook the books (Arthur Anderson consultants notwithstanding, of
course).
And finally,
don't affirm the consequentWhen
people say something like, "you can't prove a negative," what they're
probably talking about is the 3,000 year-old proof that you
cannot prove an argument true by affirming it's consequent. Consider the example:
[ 1 ] If the digital signage network is effective then we will see an increase in sales of promoted items.[ 2 ] Sales of promoted items have increased.[ 3 ] Therefore, the digital signage network is increasing sales.These
are perfectly logical-sounding statements, and a great way to "prove"
the same kinds of things that I mention in the "correlation does not
imply causation" example above. The problem is, you actually
can't
prove this argument. Even though the premises might all be true,
the conclusion is not necessarily implicated by the first two clauses.
I
think this is a good place to leave off for now. In a future
article, I'll start looking at different ways to model ROI, as well as
some techniques for getting the data you need to tweak your network's
performance.
Subscribe to comments for this article
|
Trackback
Digg this!
|
Del.icio.us
Previous Article: What does Clarity's Acquisition of CoolSign Software Mean for the Digital Signage Industry?Next Article: Calculating Digital Signage ROI: 3 Metrics that Matter
Front page of Digital Signage Insider Blog
LEGAL STUFF: The Digital Signage Insider is written by multiple authors. The author of each article is clearly identified at the start of the article. The opinions expressed in each article are solely those of the author, and do not reflect the official opinions of WireSpring Technologies, Inc. All articles are copyright © 2004-2009 by their respective author. All content besides the actual article text, e.g. surrounding branding and informational content, is copyright © 2000-2009 WireSpring Technologies, Inc. All rights reserved. Except as provided in WireSpring's
Republishing and Syndication Policy, no articles may be reproduced, in whole or in part, without WireSpring's express written consent.