The Digital Nervous System
Aug 20, 2010 Industry News, Marketing And Advertising Analytics, Statistics, Web Analytics
The web can function like a giant extension of the human nervous system. Like a spider at the center of a giant global web, you can collect and observe streams of data coming from all over the digital expanse: searches, tweets, forums, blogs, newspaper and magazine sites, press releases, Facebook and LinkedIn. Each time someone looks for or mentions your company or your product you are alerted, and you can choose in that moment to respond to it, ignore it or wait until you have more information.
Does this sound like anything you are doing now? Someone should be doing this for your company, because marketing has increasingly become an ongoing series of conversations (whether you participate in the conversation or not).
EXPERIMENT: DETECTING INSTANT RESPONSE TO MEDIA WITH THE INTERNET
There are several national TV shows that frequently have book authors as guests (the Daily Show, The Colbert Report, The Today Show, Good Morning America). The next time you find yourself in front of one of these shows when an author is on plugging their book, try the following experiment (this will work best with a show with a national audience):
1. Fire up your laptop and go to amazon.com
2. Search in the Books category for the title of the book the author is plugging on the show you are watching
3. Click to the Amazon page for that book.
4. Scroll down past the synopsis and the reviews to the section labelled Product Details. It should look something like this:
The number I have circled is the book’s current sales rank on Amazon.
5. Every few minutes while the author is on the show and for a while after that (until you bore of this experiment), hit function key f5 to refresh the page and watch what happens to the book’s sales ranking.
The rank should get better – in real time – as you are sitting there. I have done this several times when my brother-in-law has done TV appearances to promote his books, and it is amazing. Once he was on Oprah Winfrey and we saw the sales rank improve precipitously from 20-something into the top 10 while he was being interviewed.
Now imagine all the other analogous information streams there are available on the internet. If you could get the monitoring automated, just think of how quickly you will know exactly what the world thinks of your new site, your new ad campaign, your new product. Just think of what you’d be missing by NOT knowing.
EXTRA CREDIT EXPERIMENT #1 – THE TWITTER BUMP
In between rank checks you should do check in on Twitter searches for the author’s name and the book’s name. These should also pop during the author’s TV appearance.
EXTRA CREDIT EXPERIMENT #2 – THE GOOGLE BUMP
After a day or so you should go to Google Trends and see what happened to searches for the author’s name and the book’s name. These should’ve spiked on the day the author did the TV appearance. Google Trends doesn’t provide much flexibility about getting more granular (in time) data in a more real-time way, and it looks like the beta for Google Insights for Search has a latency of a couple of days.
GOOGLE EPIDEMIOLOGY – WHO KNEW THEY COULD DO THAT?
Take a look at the Google Flutrends project (http://www.google.org/flutrends) and you can see what an amazingly useful datasource this would be with access to the full detail in realtime. It turns out that counting Google searches for flu information is a quicker detector of flu epidemics than CDC reports are.
I believe it would be just as accurate in detecting other kinds of contagion sweeping through the world: fads, emerging trends, scares, rumors, accidents, disasters – this is the kind of information that businesses need to know when it involves their products, their brands, or their markets.
Tags: Amazon Book Rankings, Amazon.com, Digital Nervous System, Google Search Insights, Google Trends, Listening programs, Real-time trend detection, Social Media Montoring, Twitter
Classic GI=GO Equation Holds True for Web Analytics
Aug 20, 2010 Statistics, Web Analytics
Garbage In = Garbage Out. People who spend their working hours analyzing numbers generally come to this realization. It is true for modeling, it is true for forecasting, and it is completely true when it comes to website analytics.
The chain of events looks something like this:
1. Someone visits a website integrated with a web analytics platform like Google Analytics, Webtrends or Omniture.
2. A web page visitor either navigates to a tracked page or performs a tracked action.
3. A script is executed in the browser, sending data to the analytics platform.
4. The data is added to the datastore.
5. The data is summarized and analyzed.
Problems arise when you assume that steps 1 through 4 are happening correctly, and you move right on to looking at reports and data that come out of the process. Oddly, most site developers I have met who are instrumenting a site for web analytics consider their job done and successful if tags fire when they are expected to. They don’t look at the data as it is passed with the tags and they don’t look and see what made it into the web analytics platform’s datastore. Anything you don’t check in software development is frequently going to be wrong. If your data is wrong, then all your analysis of it will be just as wrong as the data. Again, there’s the classic equation describing this relationship:
Garbage In = Garbage Out
How do you prevent your data from being garbage? QA and debug the data, that’s how.
Before you use information coming from a web analytics solution, you should (or someone should) do these two tests:
1. Web Analytics Data Test Number One: Is the data being passed correctly?
Use a header tool of some kind to see what tags are being invoked and what kind of data they are passing to the web analytics platform. I use WASP. It shows you what kind of tag is fired when you click on navigation and site functions, and then it lists the data values the tag passes. The test is this:
Step 1: Navigate to every page in the site. A pageview should be generated for every pageview you generate and it should have the correct page name passed with it. Implementation of this is usually OK for standard HTML sites, but is error-prone for Flash sites.
Step 2: Click every function you are tracking as an action or event. See that an action or event is generated for each one you click, and that it is firing a tag that classifies it correctly – as an event, not a page view, and that the name and category that are assigned to the action are what they should be.
(Steps 3-n): Anything else you have tagged for measurement, like ad placements for an ad server, should also be clicked systematically to see that everything that is supposed to be captured about ad impressions is actually captured and passed when the tag is fired.
2. Web Analytics Data Test Number Two: Is the data making it into the database(s) correctly?
Set up your full site in staging so it will have a recognizable hostname that you can filter by in your reporting tool. Tell everyone else not to play with the version in staging for a while.
Step 1: Navigate to every page in the site in a systematic order. Do this several times. Make sure you keep track of how many times each page is viewed.
Step 2: Click every function you are tracking as an action or event. Do this several times. Make sure you keep track of how many times each action is done.
(Steps 3a-3n): Anything else you have tagged for measurement, like ad placements for an ad server, should also be clicked systematically – count the impressions and count the clicks.
Step 4: Click through every funnel you have set up, several times, all the way to the goal. If you have goals like time on site or number of pages viewed, make sure you stay long enough and look at enough pages to meet these goals.
If there are required pages in your funnels, make sure you pass through them. Again keep updating the tallies of page views and actions as you do all this.
Step 5: Wait until the data is likely to be available for reporting. Latency varies by platform. Pull reports, filtering for your hostname. You should see that the numbers of page views, actions, ad impressions, ad clicks, goals/conversions, and funnel stages matches what you did in steps 1-3. If they do not, you probably either have:
a. a tagging problem (wrong tag, misimplemented tag, redundant tags, etc.)
b. a setup problem (e.g. definitions for goals/conversions, funnels)
c. other users muddying up your data by hitting the site in staging while you are testing.
d. a more exotic and difficult problem
If this all sounds like a pain in the hindquarters, compare it to the pain of realizing that you have been reporting erroneous numbers and making business decisions based on them for months or years. Believe me, they could be so far off that you’d have been better off guessing or making numbers up. Do not trust what you cannot verify with test results, or you will have much pain and sadness in your future.
Tags: Data Validation, Debugging, Debugging Web Analytics Data, Google Analytics, Omniture, Web Analytics, Webtrends
Listing Your Way to the Finish Line
Mar 25, 2010 Statistics, TV and iTV Analytics, Web Analytics
Draft Marketing Analysis Checklist
After reading Atul Gawande’s recent book “Checklist Manifesto”, I was thinking there should be a checklist for marketing analysis. One point that Mr. Gawande makes in his book is that highly-trained specialists shun checklists because in their minds only dummies need lists. However, a majority of surgeons, while rejecting lists for their own use, would want another surgeon to use one if operating on them. This is because they know how easy it is to forget one detail in hundreds.
In marketing analysis, there are a lot of steps and a lot of things to think about, and even a smart person might drop a stitch here or there if they are not following some kind of list. I have included a rough one I dashed off quickly, in hopes that others might offer refinements, altogether better lists, or more specific versions for types of marketing programs. Here it is, have at it!
DEFINE
• SET goals/ hypotheses for program
• SELECT metrics
• CREATE a measurement plan
EXECUTE
• EXECUTE program and measurement plan
• VALIDATE raw data
• PREPARE dataset for analysis
ANALYZE
• VISUALLY EXPLORE dataset for patterns and problems
• SUMMARIZE dataset statistics
• SCORE performance vs. goals/ support for hypotheses
• LIST likely conclusions
• IDENTIFY unexpected or surprising findings
• VALIDATE likely conclusions with numerical/statistical support
• SELECT final findings
COMMUNICATE
• REPORT findings for future activity
• REVIEW findings with user community
• CAPTURE questions &issues from user community
FOLLOW-UP
• INVESTIGATE user-identified questions & issues
• IDENTIFY impact on original findings
• REPORT findings of follow-up analysis
What do you think?
Tags: Analysis, Analytics, Atul Gawande, Check, Checklist, List, marketing, Process
Conversion Optimization: It’s All About Action
Mar 21, 2010 Optimization, Statistics, Web Analytics
Optimizing your website to maximize the number of page views or visitors, while sounding reasonable, may unwittingly have you wasting marketing dollars and effort on people who won’t buy anything or participate on your website (or your advertisers’ websites) in the foreseeable future.
When you spend time and money on your site content or on audience development for your site, you want to make sure you are measuring the impact of those changes in terms of number of desired actions taken by visitors to your site, in terms of the efficiency with which you are spending resource. The key measure you are tracking on the cost side is the ECPA, or effective cost per action. If you have a small site and are passive about audience development, perhaps it makes sense to optimize to Actions Per Visit (APV), or Actions Per Daily Unique visitor (APDU). But if you are spending serious time and money then you need to track these costs and what they generate.
Lights! Camera! Actions!
Before this kind of thing makes any sense at all you have to define and start measuring the on the kinds of action you are trying to get visitors to take. Are you selling things? Are you getting paid for advertising shown on your site? Are you trying to develop leads for your business? Are you trying to get people to download something? Are you trying to get people to register or sign up? Whatever actions you want people to take on your site, they need to be measured if they are going to be the basis for your ECPA (or APV, APDU). Most of these things can be measured using Google Analytics.
In any case, once you have tagged or otherwise instrumented your site to capture your desired actions, then you can track ECPA (or APV, APDU) associated with your site.
Then when you make big changes, you can see whether they improved your site’s performance. You can measure the effectiveness of your SEM, your CPC campaigns on search engines, your affiliate programs, and your efforts to publicize your site.
Measuring Dollars Out per Dollar In
ECPA is a pretty good measure, but, it only measures efficiency on the cost side. You also want to measure the return you get in dollars and cents. You can do this (or approximate this) if you can come up with a dollar value for each of your site’s target actions, either using an average value per action type or actual value per action, then you don’t need the oversimplification that focusing only on ECPA imposes. Simply put, all actions on your site are not worth the same amount and it actually makes sense to spend more on actions that are worth more. What you really ultimately want is an ROI. I’ll talk about that in a later post.
Tags: Conversion optimization, Cost per Action, CPA, eCPA, Optimization
Math Marketing: Excellent White Paper by Dimitri Maex
Mar 15, 2010 Industry News, Marketing And Advertising Analytics, Optimization, Statistics, Web Analytics
Dimitri Maex is the Managing Director Marketing Effectiveness at Ogilvy & Mather, and the author of a fantastic white paper that is posted HERE on the WPP website . What is so great about it is that it presents exactly what most companies need to know in order to get started in harnessing the full power of quantitative marketing methods, in a package that only takes about 15 minutes to read.
He starts with the history of quantitative marketing, gives a sense of the place of “math marketing” in the current business landscape, describes the types vendors with which a company can ally, and the wraps up with how a company should organize and hire to around the new skills and challenges peculiar to the coming era of quantitatively-driven marketing.
Some nits:
I don’t like the sound of the name “math marketing”. It’s just that the math doesn’t do any marketing – people still make the decisions and integrate the insights into their work, they just use data-based metrics and statistical techniques to assist them in getting a coherent picture of what is working and what isn’t, and formulating what might work in the future. It is probably also a terrible way to brand something you are selling to execs who mostly sucked at and avoided math in school. It’s like calling it “eat your vegetables marketing”.
The section on vendors is far from exhaustive. He leaves out SEM/SEO agencies in particular, and provides only the massive brand names in most of the categories he is describing. I guess Maex works for an ad agency – so he’s not responsible for selling you on his competition – but I’d look elsewhere for a buyer’s guide.
Whatever, he is right on the money about the current state of affairs and where most companies need to go.
He wraps with a couple of lists: Seven Steps to Increased Accountability, and Seven Steps to Increased Accountability to Transformational Consumer Insights.
This is a great document for business folk who want to understand the big picture of marketing analytics and quantitative marketing techniques, and want to understand how to manage them to best effect.
Tags: Data Mining, Dimitri Maex, Doubleclick, econometric models, Google Trends, Marketing Analytics, marketing mix models, Math Marketing, Microsoft, Ogilvy & Mather, quantitative marketing
Search Volume for Analytics Ramping Up Steadily – (More Fun With Google Trends)
Jan 4, 2010 Industry News, Marketing And Advertising Analytics, Statistics, TV and iTV Analytics, Web Analytics
Just for fun, I did another Google Trends search, this time on “analytics” – adding “CRM” and “ERP” as reference points. The result seems to suggest that if you are in the business software market, that you should have an analytics offering. We’ll see, but I predict that the hot growth area in business software in 2010 will be Analytics. Searches for analytics have been steadily ramping up for the last several years, and are now at a higher level than searches for the above-mentioned enterprise business software categories.
I find it very interesting that searches for “ERP” and “CRM” have been flat for so long, but REALLY interesting that the volume of “analytics” searches surpassed them in 2009.
Tags: Analytics, Bill Seely, CRM, ERP, Google Trends, Graph, Keyword Search Frequency, Practical Marketing Analytics
Statistics: the New Plastics? – Steve Lohr/NY Times
Aug 31, 2009 Industry News, Statistics
Can you “make” yourself a statistician or do you have to be a bit of an oddball to begin with?
On August 5, 2009, the Technology section of the New York Times ran an article headlined: “For Today’s Graduate, Just One Word: Statistics ” In the article, author Steve Lohr cited interviews and research to declare Statistics to be the glamor career of the near future. I have been a practitioner of marketing analytics and modeling since the 1980s, and I have mixed feelings about this news: the smug sense that I was right all along about the future of business is tempered by my equally strong desire for my competition to remain sparse and disorganized.
Citing The Graduate in an article plugging a career in statistics is more than a little weird. The quoted screen conversation was meant to underscore the vapidity of a career choice based solely on what might make one a lot of money. “Plastics” was supposed to sound ridiculous:
Mr. McGuire: I want to say one word to you. Just one word.
Benjamin: Yes, sir.
Mr. McGuire: Are you listening?
Benjamin: Yes, I am.
Mr. McGuire: Plastics.
Benjamin: Just how do you mean that, sir?
Our values as a society have changed enough since that scene ran in theaters that the exchange might now read less like a siren song for emerging sellouts, and more like sincere advice about a great career opportunity. No matter – statistics is not easy to fake an interest in. I doubt that those who might flock into a statistics course because it was a “hot” field would stay with it long enough to get a degree much less stomach it long enough to make a pot of money.
Perhaps those in the business of training statisticians will reap a windfall over the next few years if a flood of newbies rushed headlong into the field on the promise of high-paying careers. Be that as it may, I have seen too many square pegs suffer in fruitless attempts to enter this particular round hole to believe that many could “will” themselves to master statistics and analytics. Being a “quant” has a lot more to do with your innate cognitive style than your desire for a certain salary.
There is no question that business is increasingly awash in floods of data that are being under-understood and under-leveraged, and that the skills are in short supply required to extract useful meaning and patterns from data to guide decision-making and strategy.
Tags: Bill Seely, New York Times, Plastics, Practical Marketing Analytics, Statistics, Steve Lohr, The Graduate