Facebook detailed the talks on the schedule for its 2015 @Scale conference, to be held Sept. 14 at the San Jose Convention Center in San Jose, Calif.
- Heron is now the de facto stream data processing engine at Twitter, and Karthik Ramasamy will share experiences from running it in production.
- Pinterest’s Varun Sharma will present a new in-house system his team built for serving large data sets generated through Hadoop/Hive jobs.
- David MacKenzie will share Box‘s secret sauce behind supporting its events application-programming interface, which powers its desktop sync experience, with HBase to store and serve a separate message queue for each of its 30 million-plus users.
- Microsoft‘s Arun Jayandra will discuss the experience and learnings he and the Microsoft Office 365 team have gained over the past six months leveraging Spark Streaming and Scala jobs for batch processing, with Cassandra being used to store raw events and serve real-time requests.
- Facebook’s Nathan Bronson will cover the systems at Facebook that serve the social graph, which are multitenant. Data is sharded across many machines, but each database and cache server holds all types of data and answers queries for all products. He’ll share some of the core ideas around request queuing in TAO, the company’s cache for the social graph.
- Airbnb runs both production site and data infrastructure completely within AWS, and Paul Yang will be talking about how the company migrated to a two HDFS/Hive cluster setup and built significant tooling to keep multiple petabytes of data in sync across those clusters.
- Rene Schmidt will discuss Uber‘s internal storage system, called schemaless, which has been built and deployed to store transactional data such as trip history, but with a couple of unique twists, such as being an append-only system.
- Frances Perry will talk about how Google has evolved its earlier work on batch and streaming systems–including MapReduce, FlumeJava and Millwheel–into Dataflow, a new programming model that allows users to clearly trade off correctness, latency and cost.
- Ostap Korkuna will dive into the evolution of Facebook’s monitoring system and the current challenges his team faces, including anomaly detection at scale, driving data exploration and intelligent spam fighting.
- Google’s Melody Meckfessel will discuss how her team at Google writes their systems through continuous delivery, how DevOps enables them to speed up launching features and how their engineering culture thrives.
- Facebook’s Joe Savona, Christoph Pojer and Nick Schrock will explore the structure of GraphQL servers, strategies for adopting them in an organization and the client tooling unlocked from doing so. They’ll also discuss the Relay framework and an overview of the architecture, including what’s next.
- Peter Seibel will talk about lessons learned behind Twitter’s growth from a tiny company that hosts a website built on Ruby on Rails to a slightly larger one that hosts a website built on the world’s largest Ruby on Rails application (the Monorail) and then to a company that employs more than 1,000 engineers and hosts a website and mobile apps built on hundreds of JVM-based services that run in multiple data centers.
- Google’s Rachel Potvin will outline the scale of Google’s code base, describe the company’s custom-built monolithic source repository and discuss the reasons behind choosing this model of source control management.
- Facebook’s Paul Saab will discuss the growth of IPv6 on Facebook and how it will affect app developers moving forward.
- Microsoft’s Jonathan Bergeron and Jakub Grzmiel will discuss how engineering systems that support this rapid time-to-market also enable the company’s developers to easily scale out to a multitude of experiences from mobile, Xbox, PC and tablet for international markets, and then send back data in near real-time to allow further refinements of the shipped idea that same evening.
- GitHub‘s Ben Ogle will join Facebook’s Ryan Bergauer and Jess Lin to talk about increased difficulties in supporting a number of engineers and a diverse selection of languages with off-the-shelf development tools and Nuclide, an integrated development environment designed to address these problems and provide a single, scalable editing environment for mobile, Web and more.
- Twitter’s Jess Garms will talk about how his team at Twitter builds mobile apps for unreliable networks.
- Facebook’s Jonathan McKay and Nate Schloss will present with Owen Campbell-Moore from Google to discuss the work Facebook has done with Google, UC and Opera by using private and public APIs to bring the benefits of native apps to everyone.
- Clement Genzmer and Greg Moeck will share some of the innovative tooling Instagram has developed to monitor central processing unit consumption during scrolling, larger strategies for optimizing user interfaces and “tips and tricks” they’ve uncovered deep down in iOS and UIkit to harness the power of every core.
- Martin Destagnol will discuss how Box transitioned its mobile strategy from building primarily end-user apps to a state where mobile software-development kits and mobile platform components come first, highlighting findings and challenges along the way.
- Peter Cottle and Brian Sa will talk about how Facebook leverages the social graph to reach millions of people affected by natural disasters and deliver news of their well-being to their families and friends, and generally how personalized announcements across the Web and mobile clients can be delivered quickly and efficiently.
Readers: Do you plan to attend 2015 @Scale?