TikTok to Rely More on Machine Learning to Surface Content Violations in Specific Categories

TikTok brought several systems it has been testing in markets around the globe to identify and remove content that violates its policies to Canada and the U.S. Friday. Head of U.S. safety Eric Han explained in a blog post, “Like most user-generated-content platforms, content uploaded to TikTok initially passes through technology that works to identify and flag potential policy violations for further review by a safety team member. If a violation is confirmed, the video will be removed and the creator will be notified of the removal and reason and given the opportunity to appeal the removal. If no violation is identified, the video will be posted and others on TikTok will be able to view it.” Han said that over the next few weeks, the video creation platform will begin using technology to automatically remove some types of content that violates its policies immediately upon upload, with automation reserved for categories where TikTok’s technology has the highest degree of accuracy, such as: adult nudity and sexual activities; illegal activities and regulated goods; minor safety; and violent and graphic content. As always, creators will be able to appeal removals of their videos directly via the application or report potential violations to TikTok for review. Han wrote, “In addition to improving the overall experience on TikTok, we hope this update also supports resiliency within our safety team by reducing the volume of distressing videos moderators view and enabling them to spend more time in highly contextual and nuanced areas, such as bullying and harassment, misinformation and hateful behavior.” TikTok also detailed changes to how it notifies users of community guidelines violations, saying that the new system it put in place counts violations accrued by users and is based on the severity and frequency of those violations. Han detailed the process: In the case of a first violation, TikTok will send a warning to the creator in its app, unless one of its zero tolerance policies is violated, such as posting child sexual abuse material, in which case the user will be banned automatically. After the first violation, TikTok may: Suspend an account's ability to upload a video, comment or edit their profile for 24 or 48 hours, depending on the severity of the violation and previous violations.Restrict an account to a view-only experience for 72 hours or up to one week, meaning that the account can’t post or engage with content.After several violations, a user will be notified if their account is on the verge of being banned. If the behavior persists, the account will be permanently removed. Han said that in testing in Canada and the U.S. over the past few weeks, more than 60% of people who received a first warning did not have further violations. He concluded, “While we strive to be consistent, neither technology nor humans will get moderation decisions correct 100% of the time, which is why it's important that creators can continue to appeal their content's or account's removal directly in our app. If their content or account has been incorrectly removed, it will be reinstated, the penalty will be erased and it will not impact the account going forward. Accrued violations will expire from a person's record over time.”

Platforms

TikTok to Rely More on Machine Learning to Surface Content Violations in Specific Categories

The video creation platform also overhauled the way it notifies creators who violate its community guidelines

David Cohen

About

Subscriptions

Events

Publications

WORK SMARTER - LEARN, GROW AND BE INSPIRED.

Subscribe today!

David Cohen