Facebook Shed Some More Light on the People Behind Its Content-Review Process

Facebook used the latest installment of its Hard Questions series to remove some of the mystery behind the human side of its content-moderation process. Vice president of operations Ellen Silver said in her Hard Questions post that ensuring the safety of Facebook’s content reviewers is one of the reasons why details have been scarce, writing, “As we saw with the horrific shooting at YouTube’s headquarters earlier this year, content reviewers are subject to real danger that makes us wary of sharing too many details about where these teams are located or the identities of the people who review. The day-to-day challenges our reviewers face and the obscurity of their work often leads to confusion about what our reviewers actually do and whether they’re safe doing it.” Silver reiterated some previously shared details: Facebook will double the number of people working on safety and security to 20,000 by year-end, including 7,500 content reviewers—full-time employees, contractors and employees from partner companies. She added that “language proficiency is key,” and that Facebook has reviewers in the languages it supports available 24/7, as well as “translation companies and other experts” in the event they are needed. Silver also pointed to an emphasis on culture, as well as language, writing, “We want to hire Spanish speakers from Mexico—not Spain—to review reports from Mexico, as it often takes a local to understand the specific meaning of a word or the political climate in which a post is shared.” She continued, “Beyond these qualities, it’s important that we hire people who can handle the challenge these jobs pose. This work isn’t for everyone, so we look for candidates who we think can cope with violent or troubling imagery. Sometimes we explore whether a past trauma might be triggered by such work. And of course, we meet all applicable local employment laws and requirements.” Silver detailed the three areas of training that content reviewers—full-time, contractors and employees of partner companies—go through: Pre-training, which includes what to expect on the job. Each hire also learns how to access resiliency and wellness resources and gets information on how to connect with a psychologist when they need additional support. Hands-on learning, including a minimum of 80 hours with a live instructor, followed by hands-on practice using an actual replica of the system so that new hires can practice in a “real” environment. Following hands-on training, reviewers get a report highlighting areas where they’re applying policies consistently and accurately and where they need more practice. Ongoing coaching: Once hired, all reviewers get regular coaching, refresher sessions and policy updates. Once a Facebook user reports a piece of content, it is automatically routed to a content-review team, based on language or type of violation, ensuring that someone with specific training in that particular area conducts the review. Silver said each reviewer is assigned a queue of reported posts to evaluate individually, adding, “Sometimes this means looking just at the post itself to determine whether it should be allowed—such as an image containing nudity. But other times the context is key, and so additional information, like comments on the reported post, is provided, as well.” She shed more light on the process, writing, “Our auditors are even audited on a regular basis. In addition, we have leadership at each office to provide guidance, as well as weekly check-ins with policy experts to answer any questions. A common misconception about content reviewers is that they’re driven by quotas and pressured to make hasty decisions. Let me be clear: Content reviewers aren’t required to evaluate any set number of posts—after all, nudity is typically very easy to establish and can be reviewed within seconds, whereas something like impersonation could take much longer to confirm. We provide general guidelines for how long we think it might take to review different types of content to make sure that we have the staffing we need, but we encourage reviewers to take the time they need.” Finally, Silver discussed resources that are available to all reviewers, including mental health resources such as trained professionals for individual and group counseling. The work environment is factored in, as well, with Silver writing, “There’s a misconception that content reviewers work in dark basements, lit only by the glow of their computer screens. At least for Facebook, that couldn’t be further from the truth. Content review offices look a lot like other Facebook offices. And because these teams deal with such serious issues, the environment they work in and support around them is important to their well-being.”