How NPR’s Debate Night Fact-Check Went Down

A real-time transcript and about 30 staffers, to start.

NPR’s fact-checking operation for Monday’s presidential debate was pretty epic. Instead of putting out periodic updates of selected sentences or a post-debate analysis, NPR went for a real-time fact-check of a scrolling script of the debate being transcribed as the debate was occurring.

The transcription, explains NPR ombudsman Elizabeth Jensen, was outsourced and automated but proofread and checked over by staff. NPR’s fact-checking crew was about 30-people strong, comprised of reporters, editors and visual staff. “We reached out across desks to policy and beat experts and editors from business to foreign affairs to education and created a shared document that everyone who wanted to be involved could do so in an easy way,” NPR lead political editor Domenico Montanaro told Jensen over email.

In the end, 74 statement were fact-checked and/or contextualized. In terms of what was chosen, “the editorial was organic,” explained Montanaro. “Like viewers at home, when we heard something that made us go, ‘Is that true?’ Or ‘That’s not true.’ Or ‘well that’s not the whole context,’ our reporters were on it,” he wrote.

The breakdown of facts checked included nine of Hillary Clinton‘s, 33 of Donald Trump‘s and 32 contextual annotations (11 for Clinton statements, 21 for Trump). “The vast majority of fact-checks on Trump concluded that he was wrong,” writes Jensen. “By our count just a handful of Clinton’s statements were judged wrong.”

Jensen asked Montanaro about the imbalance between the number of Trump vs. Clinton fact-checks. “What’s fair isn’t always equal,” he wrote. “We certainly are always cognizant of providing balance and focusing on both candidates, but if the numbers show that one candidate was fact checked more, it might be because that candidate said more things that needed fact checking.”

The scale and scope of the undertaking made it something of an experiment for NPR–not just in terms of the work it required, but in terms of who would be interested in the fact-check and in what numbers.

And the numbers, according to Nieman Lab, were good, very good:

NPR.org saw 7,413,000 pageviews from 6,011,000 users, and 22 percent of visitors to the page stayed all the way to the end, according to a spokesperson. Monday was NPR.org’s highest traffic day ever, with over 5 million visits. The annotated live transcript also saw 70 percent of its traffic coming from mobile devices, and people coming from mobile actually stayed four minutes longer than people coming from desktop, according to Eads.