The Skinny on Data Collecting

Talk about counting eyeballs. Usually, when it comes to the presentation of Web data, the first things monitored are page counts and ad impressions. But in a new report published by online auditing firm ABC Interactive, the monitors themselves are the ones being monitored. And, according to ABCi, the numbers didn’t always add up.

In the study, ABC Interactive reported that unique user counts, as well as counts of page impressions, were significantly underreported by two “leading” research companies. Both Net research firms use panel-based measurement methods, which, according to Rolling Meadows, Ill.-based ABCi’s Reconciliation Analysis service, produce data that has been inconsistent with those figures culled from companies’ own census-based Web site log file records.

ABCi concluded that between 40 and 57 percent of unique user and page impression counts were underreported by the two research companies. Dick Bennett, senior vp of audit services at ABC Interactive, said that while there is “less than total perfect measurement in both [panel-based efforts and Web site log file records],” Web file census records capture “every file that is exchanged between the servers and the browsers … The panel, obviously, is only working with the individual transactions that the members of the panel are having with those Web sites. So there always are certain issues with latency, with caching, etc.”

Bennett said that there was evidence that a “number of Web sites believed that they [were] being inaccurately measured by the panel companies.”

Rebecca Young, vp of marketing for Milpitas, Calif.-based Nielsen NetRatings, disputes ABCi’s analysis. Young said that far from companies claiming substantial discrepancies between independent, panel-based data and their own Web file records, NetRatings “found that through reconciliation analysis of some of our customers that they’ve done themselves, we’re actually very, very close to the server logs.”

Young described NetRatings’ method of data measurement as “incredibly rigorous,” and noted that the size of the company’s panel was now at about 225,000 worldwide. She also said that methods of data measurement weren’t an “either or” proposition, that there was room for both panel-based results and Web file consensus data.

“A lot of the companies will use the data in combination,” Young said. “All companies are looking at their server logs.”