Twitter Slammed For Low-Paid 'Human Data Analysts' - But Are Critics MisInformed?

According to some sources, these workers MUST be low paid – and correspondingly, exploited somehow. But we think these critics may be misinformed.

If you saw the post on Business Insider today, the headline screams (subliminally) “Twitter is evil.” It read: Twitter Uses Army of Low-Paid Workers to Manually Sort Hashtags for Advertisers. Brings to mind some poor, exploited third-world folks hunched over computers, eyes propped open by a devious machine as they listlessly swat at a keyboard, right?

Well, you may not have read through the entire blog post detailing how Mechanical Turk operates (because it’s very long and attention spans are very small), but let’s summarize some key parts that may remove that third world nightmare scenario from your mind.

Twitter sends queries to Mechanical Turk people, folks they call “judges” and has them sort tweet data in to categories, find Twitter handles, perform Google searches and such. This helps Twitter understand what the heck people are tweeting about when they don’t know. And this helps Twitter provide better search results for users and better ads for businesses.

Now back to the “judges.” Twitter needs “fast and reliable results” so they use “a small custom pool of Mechanical Turk judges to ensure high quality.” Did you catch that? A small custom pool of judges. THAT sounds like they need experience to do this job. Experienced workers probably aren’t cheap.

But there’s more. These judges “work virtually all day. For many of them, this is a full-time job, and they’re geographically distributed, so our tasks complete quickly at all hours.” And they “are culled from the best of Mechanical Turk, they’re experts at the kinds of tasks we send, and can often provide higher quality at a faster rate than what even in-house judges provide.” So they work from home and on their own schedules? Nice.

Now to the part so many are focusing on: How much Mechanical Turk workers are paid. If you look at the Mechanical Turk website, you’ll see jobs listed that pay from .01 to $40 USD per “hit.” And if you search for jobs that mention “Twitter” or “tweet” you’ll find a short list of tasks asking for things like “find this company’s Twitter handle” for .05 or “write tweets about pop culture” for .08 per tweet. How long would either of those tasks take you?

But those aren’t the jobs posted by Twitter. Twitter (undoubtedly) isn’t listing its jobs there, but just sourcing candidates when it needs to bring on more from time to time – they don’t have to, they have a custom pool of judges from Mechanical Turk, remember? And they have their own communication protocols:

We have several forums, mailing lists, and even live chat rooms set up, all of which makes it easy for judges to ask us questions and to respond to feedback. Our judges will even give us suggestions on how to improve our tasks; for example, when we run categorization tasks, they’ll often report helpful categories that we should add.

But you probably still have concerns about these “low paid Amazon workers.” Maybe they’re just paid crappy in private? Well, did you catch the little video at the end of the Twitter Engineering blog post where they thanked a top “Turker” called workasaurusrex?

Special thanks in particular to our amazing Turker, workasaurusrex, the musician and silky smooth crooner who brought the masterpiece together.

Many thanks to the Revenue and Storm teams, as well as our Turkers, for their help in launching this project.

Posted by Edwin Chen and Alpa Jain

Workasaurusrex is @Chipperr7 on Twitter:

And that site he lists in his bio, CloudMeBaby “is dedicated to improving efficiency and maximizing money making potential in cloud based work environments such as Amazon Mechanical Turk, oDesk, elance and other online job opportunities.”

Apparently, it works:

Not what you expected, hmm? We reached out to confirm this figure was for work as a judge through Mechanical Turk, but he didn’t immediately respond. (If/when he does, we’ll let you know.) Either way, if you check out the CloudMeBaby site and start to really think about this whole process, you’ll likely conclude that these “low paid Amazon workers” aren’t – they’re smart. And Twitter is too. These tasks are too important to Twitter’s bottom line to imagine otherwise.

Update: workasaurusrex clarified that his “judge” reference did not refer to Turk work (for that, he’s a “turker” regardless of how anyone else classifies it). He would not divulge how much he is paid, but did not strike me as displeased. 

(Hand on data image from Shutterstock)