Facebook recently took users behind the curtain for an in-depth look at the engine that powers the social graph: TAO. Whenever you check into a place, comment on a friend’s status, or like a photo, TAO knows.
TAO, which stands for The Associations and Objects is the motor that keeps Facebook’s graph going, learning about relationships between users.
Facebook illustrated what TAO does and thoroughly explained its function:
Facebook puts an extremely demanding workload on its data backend. Every time any one of over a billion active users visits Facebook through a desktop browser or on a mobile device, they are presented with hundreds of pieces of information from the social graph. Users see News Feed stories; comments, likes, and shares for those stories; photos and check-ins from their friends — the list goes on. The high degree of output customization, combined with a high update rate of a typical user’s News Feed, makes it impossible to generate the views presented to users ahead of time. Thus, the data set must be retrieved and rendered on the fly in a few hundred milliseconds.
This challenge is made more difficult because the data set is not easily partitionable, and by the tendency of some items, such as photos of celebrities, to have request rates that can spike significantly. Multiply this by the millions of times per second this kind of highly customized data set must be delivered to users, and you have a constantly changing, read-dominated workload that is incredibly challenging to serve efficiently.
If you’re interested in learning more about TAO, the team behind it recently published a paper presented at the 2013 USENIX Annual Technical Conference in San Jose.
Teaser image courtesy of Shutterstock.