Facebook Open-Sources Warp Preprocessor For C, C++

By David Cohen 

CodeOrange650The Facebook open-sourcing train continued to roll on with Friday’s announcement that Warp, a preprocessor for the C and C++ languages, has been open-sourced.

Warp was written by Walter Bright in a joint project with Facebook, and Andrei Alexandrescu, a research scientist with the social network, said in a post on its engineering blog announcing its open-sourcing:

Companies that work with large C++ code bases pay close attention to build times, as they have a major impact on engineers’ productivity. At Facebook, we have a sophisticated build system that uses distributed compilation in conjunction with several cache layers backed up by memcache. Part of this architecture relies on caching preprocessed files.

Conventional wisdom has it that preprocessing time is a negligible part of building a C++ binary. C++ is notoriously difficult to parse, which means that C++ parsers tend to be large and slow compared to their counterparts for other languages. Code generation is also quite the time sink, especially if you’re using optimizations. Finally, linking is a large serial step at the end of each build, and using the gold linker only helps so much. Therefore, it would seem that the lowly task of going through the included files and expanding macros would take a comparatively short time and be essentially impossible to improve upon.

Not so! Replacing gcc’s preprocessor with Warp has led to significant improvements of our end-to-end build times (including linking). Depending on a variety of circumstances, we measured debug build speed improvements ranging from 10 percent all the way to 40 percent, all in complex projects with massive codebases and many dependencies. That’s not per-file speed, but rather global times measured for scenarios like “build after changing a header file.”

Bright goes into far more detail about Warp in an interview with Alexandrescu in the blog post.

Image courtesy of Shutterstock.