Winner of Twitter’s Algorithmic Bias Bounty Competition Speaks Out

Acknowledgment is the first step of recovery, but it's still just a step. Twitter's image cropping algorithm favors lighter, slimmer and younger faces, researcher Bogdan Kulynych found. Kulynych went on to warn that bias algorithms are just one way tech can be harmful. Kulynych, a graduate student at public research university École Polytechnique Fédérale de Lausanne in Lausanne, Switzerland, was named the winner of Twitter’s algorithmic bias bounty competition at the DEF CON 29 security conference in Las Vegas. After his victory, he took to the platform to express mixed feelings about his findings. The competition, kicked off in July, was Twitter’s way of enlisting the help of the research and hacker communities to address issues with its image cropping algorithm that initially surfaced last October. How Deep Learning AI Can Transform Challenger Brands Into Cutting-Edge Leaders The social network responded to reports that the algorithm favored white faces over Black faces, and even white dogs over black dogs, with chief technology officer Parag Agrawal and chief design officer Dantley Davis writing in a blog post at the time, “While our analyses to date haven’t shown racial or gender bias, we recognize that the way we automatically crop photos means there is a potential for harm. We should’ve done a better job of anticipating this possibility when we were first designing and building this product.” Twitter pledged to “add further rigor” to its testing, share its findings and explore ways to open-source its analysis. Testing the algorithm Kulynych’s approach was to artificially generate faces with different features and feed them through the cropping algorithm to see what it focused on, and his use of artificial faces enabled him to generate examples that were nearly identical but differed in age, gender and skin tone. He found that the algorithm favored lighter faces over darker ones, slimmer faces over wider ones and younger faces over older ones. Rumman Chowdhury, director of Twitter machine learning ethics, transparency and accountability team, said during the DEF CON 29 event, as reported by The Guardian, “When we think about biases in our models, it’s not just about the academic or the experimental … but how that also works with the way we think in society. I use the phrase, ‘life imitating art imitating life.’ We create these filters because we think that’s what ‘beautiful’ is, and that ends up training our models and driving these unrealistic notions of what it means to be attractive.” Ethical concerns about algorithms Referring to the algorithmic bias bounty competition, Chowdhury and product manager Jutta Williams enthused about the need for "proactive and collective identification of algorithmic harms," for which Twitter can set a precedent for the industry. While Kulynych praised the company for not only seeking outside help but for rewarding those efforts, as well, he expressed notes of caution in tweets after Twitter notified him of his victory in the challenge and $3,500 reward. "I have mixed feelings. First, algorithmic harms are not only ‘bugs.’ Crucially, a lot of harmful tech is harmful not because of accidents, unintended mistakes, but rather by design," Kulynych added. "This comes from maximization of engagement and, in general, profit, externalizing the costs to others. As an example, amplifying gentrification, driving down wages, spreading clickbait and misinformation are not necessarily due to ‘biased’ algorithms.” He continued, “We should not forget that ‘algorithmic bias’ is only a part of a bigger picture. Addressing bias in general and in competitions like this should not end the conversation about the tech being harmful in other ways, or by design, or by fact of existing. We should resist the urge of sweeping all societal and ethical concerns about algorithms into the category of bias, which is a narrow framing even if we talk about discriminatory effects.” Twitter's journey Before the competition, the platform has been taking steps in response to the bias allegations from October. After analyzing its models, Chowdhury shared the company’s findings in a blog post in May, it found there was an 8% difference favoring women over men, a 4% difference favoring white individuals over Black individuals, a 7% difference favoring white women over Black women and a 2% difference favoring white men over Black men. Chowdhury also pointed out Twitter’s release in May of non-auto-cropped images on Android and iOS, as well as image previews in the tweet composer field so that users can see what their tweets will look like before publishing them. Twitter reshared its saliency model and the code used to generate a crop of an image given a predicted maximally salient point for the challenge, in which participants were asked to arrive at their own assessments, via both quantitative and qualitative methods.