I just did a rapid search on the Yahoo User Groups with the term “ComScore“.
Just to see if the discussions we’ve been witnessing currently had crossed our paths before.
And they did: 160 results, dating back to the 7th of August 2004. In the last 10 results, I’m amused to find names such as Jim MacIntyre, current WebSideStory/Visual Sciences CEO and working already in January 2005 for VS, Google’s Brian Clifton a voice back then from Omega, Fred Kuu and … hum, Sage metrics turned Omniture, for some.
So, the discrepancies in numbers is not a new debate. Normal, it’s not the same way of counting 😮 No surprises there.
What started it off this time?
Apparently, and I’m still picking this up from the Yahoo User Group to start with, on April 16th, ComScore announced through a press release that “Cookie-Based Counting Overstates Size of Web Site Audiences. Frequent Cookie Deletion by 3 out of 10 U.S. Internet Users Leads to Overstements in Audience Sizes by a Factor of High as 2.5“.
Micheal Wexler, Eric Peterson & Joe Wilson made sure the announcement was brought forward to the Yahoo community and it spurred of comments, a lot!
On the Yahoo user groups, discussions lead by different people amongst which, Eric Peterson who blogged a couple of times about this on his own blog as he was quoted in the press release and volunteered to find out more about the method used, without his pony tail. Our hero!
Reactions ranged from “so what? I already knew this” to “ho my, I’m calling a crisis meeting“. Some discussions spurred off onto 1st party cookies being “more accurate” than 3rd party ones and the imminent ComScore IPO. Apparently, Web Analysts are defiant about coincidences 😉
Personally, I’m of the first kind and count my blessings that I don’t work for a media buying agency. I don’t like the term “reach” and mistrusts “random” samples. Being an econometrist, I know that random is just a theory (like economics) and we can only come as “close” as we can get so we’re always open to attacks in the lines of “I don’t believe this data“. Add to that that the notion of “accuracy”, like justice, remains relative in my eyes and is often influenced by the position of the camera you hold upon the subject.
Different positions upon what appears to be the same metric, i.e. unique visitors, hence the IAB press release some 4 days later demanding for a third party audit of methods that count the size of web audiences. And transparency in the method. Ok, that’s a good start.
3 days later, ComScore comes out with a new press release welcoming the initiative and so does Nielsen/Netratings. Everyone happy?
I for one am amused but not reassured.
I’m happy they are talking but to be quite frank, I’d be happier if someone of the Web Analytics Association was also in on the loop and hope there is. And as someone stated on the Yahoo user groups, it’s not only a technical problem, it’s more a political problem. I can’t help but agree on that one.
Some examples from Swedish and French traffic measurement figures were quoted on the Yahoo user groups but fact remains, when the IAB -and therefore mainly media buying agencies, I’m almost sorry to say, at least here in Belgium- talks about unique visitors, they’re not speaking my web analytics lingo. Add to that that across Europe, I’m not really confidant that the OJD (french) and the CIM (belgian) actually use the same kind of tools and you’ve already got a pan European comparison problem on your hands, even if you just look at trends.
Last but not least, (good) web analytics tools have been counting unique visitors through 1st party cookies for some years now, and you’re sure you’ll end up with questions such as the last post I’ve seen on the Yahoo User Group:
When large portals report that they have XYZ# of uniques. Are they referring to unique visits to the site or unique visitors to the site?
The two numbers are very different and I understand the difference, but would like to know what the industry standard is when some refers to their “uniques.”
No, thank you Nicole, for the question! and please allow me to add, you’re not alone!
What are we actually trying to do here? why do we need these numbers and why should they be aligned? that’s the bottom line after all, right? and indeed, you can’t go back to your affiliate partner and tell him you’ll slash his bill in 2 as they are wrongly counting their unique visitors, can you now? If indeed, you’re paying per unique visitors. Eyeballs, right? Funny, Düsseldorf also showed up branding metrics. I attribute that to market immaturity, to be quite blunt. But hey, that’s me and I’m still waiting to be convinced by my media friends. Buy me a coffee and/or an Amaretto and I’ll lend you my ear 😉
But what if you’re paying per conversion. Would this still matter? I suppose so, as you’d be trying to look first at the larger population before diving into a possible long tail and actually spending energy (and brains hopefully) at optimising your traffic acquisition investments. Which can bring you to rules such as “any conversion should cost less than 1 euro”, pushing also media agencies at working harder for their buck/euro/yen/danish crown… After all, we’ve already moved from CPM to CPC based pricings, why not go further?
Because conversion means different things to different people. Yeap, that’s where it hits the nail. At some point it boils to down how much you’re actually willing to pay, by your acceptable standards and that requests maturity, as our friend from Pioneer so elegantly demonstrated in Germany.
This however does not mean we should not at least strive for some kind of comparable view and for starters, I’d settle for a pan European one currently based on 3rd party cookies as a meeting in France last week reminded me. It would be good for Europe, help pan European use of the Internet, IMHO.
I also welcome the IAB in their discussions and would urge them to add the WAA to these discussions, including both practitioners and vendors to come to something more comprehensible to all. The IAB and the WAA both hold a focus on training so, work together. The fact also that the IAB used the word “auditing” also reminds me of a comparison I made to René some months ago between the Big4/5 and Web Analytics but one thing at a time. After all, if the accountants can do it, so can we or should be worry about Enron type scandals? lol
Last but not least, I understand the need for 3rd party cookies in order to calculate “reach” and have seen the consequences of 3rd party cookies when crunched by Web Analytics tools -my best record is 65% rejection rate in a B2B environment for the moment. There seem to be other alternatives in terms of counting as Oliver pointed out in Germany, ranging from HTTP headers (I also saw that at the Emetrics Summit in Washington last fall) to MAC addresses. Yes, I know boo if we talk about privacy, I quite agree.
The first step is education and transparency so that we can understand what we’re comparing and put the gathered information into context. I’m confident there are plenty of smart people that, with time, will figure out what would be best and how this industry can evolve. For the time being, let’s just make sure that we talk about the same things and that when I tell a client “be aware of unique visitors” and don’t take them for granted, they actually do.