The massive size of the data in large graph processing requires distributed processing. However, conventional frameworks for distributed graph processing, such as Pregel, use programming models that are well-suited for scalability but inconvenient for programming graph algorithms. In this talk, we demonstrate how to use Green-Marl, a Domain-Specific Language for graph analysis, to describe graph algorithms intuitively and extend its compiler to generate equivalent Pregel programs. Using the semantic information exposed by Green-Marl, the compiler applies the same kinds of transformation rules that programmers apply when manually implementing graph algorithms with Pregel. Our experiments show that the Pregel programs generated by Green-Marl compiler perform similarly to native Pregel implementations of the same algorithms. The compiler is even able to generate a Pregel implementation of a complicated graph algorithm whose native Pregel implementation is very challenging.

The first paper describing Green-Marl was published at ASPLOS 2012. http://arsenalfc.stanford.edu/kogroup/papers/asplos2012.pdf We have extended our compiler to now generate Giraph code in addition to our c++ backend. The source code is available here: https://github.com/stanford-ppl/Green-Marl