Web Execs Question Metrics

NEW YORK It's been a long-standing tradition among Web publishers to occasionally gripe about their numbers. But in the last several months, what was mild grumbling has become a cacophony of complaints, as several top publishers call into question the validity of the traffic data being provided by the industry's major third-party metrics firms: Nielsen//NetRatings and comScore Media Metrix.Now the Interactive Advertising Bureau is pressuring the two firms to increase their transparency and improve their methodologies—just as both prepare to undergo long-awaited Media Rating Council-led audits. IAB insiders say an outside research firm such as DJG Marketing may be hired by the end of the year to examine the soundness of Nielsen's and comScore's data. At issue are standard traffic metrics for ad-supported Web sites: unique users and page views. Though sites have long contended that the numbers Nielsen and comScore provide have always been lower than what they record on their own servers, many now say that the disparity is widening. "It's always been sketchy," said Johanna Steen, MSNBC.com research manager. "Now the variation is really growing." For example, MSNBC's site logs sometimes show a 30 percent difference in page views. "We'll report a billion page views and Nielsen will report 600 million," she added. "For a site as large as ours, we should be more consistent. It's definitely a concern for us."Several top publishers, who spoke on the condition of anonymity, said that over the last six months to a year, Nielsen's and comScore's numbers have become less reliable, often showing little correlation with site-reported traffic ups and downs. Some contend that both companies' panel recruitment efforts have suffered, perhaps due to budget cuts. And many complain that it's difficult to identify the source of such variances. "They are not very transparent," complained one publisher.While outgoing IAB president Greg Stuart flatly denied that the organization plans to seek the help of an outside consultant, he did not hesitate in identifying metrics as a growing problem. "The data from the panels seems to be increasingly flaky," he said. "I'm concerned that they are not keeping pace with the speed that the industry is becoming mainstream."As the industry grows rapidly, more publishers are scrutinizing individualized traffic reports produced by Web analytics firms like Omniture. That's indirectly raising the expectations for Nielsen and comScore. "I'd like to see a product that tracks every page," said MSNBC's Steen.Yet that's not what panels are designed to do, explained Mainak Mazumdar, vp, product marketing and measurement science for Nielsen//NetRatings (an affiliate of VNU, which owns Mediaweek). "We count people, not pages," he said. Mazumdar explained that since those services don't employ cookies, they often count the same user twice, resulting in 20 percent to 30 percent duplication. Regarding page views, Mazumdar pointed out that Web analytics companies record every single page a site serves, including pages that are pushed at users (like pop-ups), registration and thank-you pages, and even pages that are accessed by robots. "We want to measure pages that the user is actually trying to get to," he said. Still, Mazumdar acknowledged that he has gotten "more calls lately from publishers."For its part, comScore says that a few vocal publishers are making a lot of noise about isolated problems. Lynn Bolger, comScore's exec vp, agency development, flatly denied charges of weaker panels or budget cuts: "Those are simply not substantiated complaints...That's pure conjecture." Bolger added that while there will always be unhappy clients, "more often than not...we come to a systematic understanding of what the differences are."That responsiveness, while appreciated, points to a problem. Dan Agronow, chief technology officer, Weather.com, said he's often pointed out traffic discrepancies to both services, which eventually get corrected. "It concerns me that it takes vigilance to make sure that their methodology is consistent," he said.Both vendors say these issues will be addressed via audits (each is currently in a "pre-audit" phase, says the MRC). Yet many say that getting both to agree to that has taken way too long. "I'm not happy with the speed of the process there," said Stuart. While Nielsen and comScore are taking heat from some quarters, Ted Smith, senior vp of research and market intelligence at CNET, said that he has not seen any discernable decline in either company's data over the past two years. Smith credited both for doing an "excellent job" in delineate macro traffic trends. However, Smith said they fall short when providing more granular site info: "We have a lot less confidence in the monthly data." Media buyers, who ostensibly have a vested interest in metrics, are being less vocal. That's because, unlike in the TV world, agencies don't buy inventory based on third-party data, but rather purchase guaranteed ad impressions from sites, which are typically tracked by third-party ad servers like Doubleclick or Atlas. Why turn to projected numbers when you have access to real ones? "Third-party numbers are more for new advertisers," said Jeff Lanctot, vp, general manager of Avenue A/Razorfish. "We view them as a secondary after our own internal data."Tracking future online activity will only get more complex. Some advocate melding sites' log data with panel-based data, something Nielsen is trying. "The audience measurement firms should think about overlaying server data with their panel data to arrive at a more refined set of statistics," said Vivek Shah, Time Inc.'s business and finance network. "Auditing site-server data and third-party panel data absolutely makes sense."Yet many wonder whether using panels makes any sense in a medium that is so trackable. Said one publisher: "Why are we using television methodology?"

Brand Marketing

Web Execs Question Metrics

Mike Shields

About

Subscriptions

Events

Publications

WORK SMARTER - LEARN, GROW AND BE INSPIRED.

Subscribe today!

Mike Shields