FOSDEM is the biggest free and non-commercial event organized by and for the community. Its goal is to provide Free and Open Source developers a place to meet. No registration necessary.

Claudio Martella
Day Sunday
Room AW1.125
Capacity 76
Start time 09:30
End time 10:15
Duration 00:45
Track Graph Processing Devroom

Apache Giraph: distributed graph processing in the cloud

Web and online social graphs have been rapidly growing in size and scale during the past decade. In 2008, Google estimated that the number of web pages reached over a trillion. Online social networking and email sites, including Yahoo!, Google, Microsoft, Facebook, LinkedIn, and Twitter, have hundreds of millions of users and are expected to grow much more in the future. Processing these graphs plays a big role in relevant and personalized information for users, such as results from a search engine or news in an online social networking site.

The Apache Giraph [1] project is a fault-tolerant in-memory distributed graph processing system which runs on top of a standard Hadoop [2] cluster and is capable of running any standard Bulk Synchronous Parallel (BSP) operation over any large generic data set which can be represented as a graph. Apache Giraph is a loose implementation of Google Pregel but can be added to any Hadoop job pipeline as a normal MapReduce job. Giraph entered the ASF Incubator in July 2011, where it has enlisted the aid of committers from Yahoo!, Facebook, LinkedIn, and Twitter.

The talk will describe why running iterative MapReduce jobs for graph processing is not well suited for typical MapReduce jobs, introducing the reason why Google designed Pregel at first place. Next, the BSP model and how it is applied to graph processing will be explained. The last part of the talk will be dedicated to Apache Giraph, with a description of the programming model (i.e. the API, some typical examples such as PageRank and Single Source Shortest Path) along with a technical overview of how the architecture of Giraph works and how it leverages the Hadoop infrastructure.

Concurrent events:

When Event Track Where
09:00-09:40 LTE is here, and ModemManager is (almost) ready for it Telephony and Communications H.2213
09:00-09:45 Rudder - configuration management benefits for everyone Configuration and Systems Management K.3.601
09:00-10:00 Writing a Wayland Compositor K.3.401
09:10-09:55 Xonotic: The road to 1.0 Open Source Game Development AW1.120
09:15-09:55 GNOME 3.4 accessible: Status, news, future CrossDesktop H.1308
09:15-10:00 MINIX 3 and BSD BSD K.4.201
09:30-09:55 USB redirection over the network Virtualization and Cloud Chavanne
09:30-10:00 Take a small REST, Simple approaches for REST in smalltalk Smalltalk AW1.126
09:30-10:00 Why apps start slowly on Linux and what to do about it Mozilla UD2.218A
09:30-10:00 Introduction to the NOVA kernel API Microkernel OS K.3.201
09:35-09:55 Advanced Moose Techniques Perl AW1.121
09:35-10:00 Sphinx User stories MySQL and Friends H.1309
09:45-10:25 Asterisk 10: New Features, New Testing Telephony and Communications H.2213
10:00-10:15 XQuery 3.0 Rocks Lightning Talks Ferrer
10:00-10:25 MySQL HA reloaded - old tricks and cool new tools to guarantee high availability to your MySQL Servers MySQL and Friends H.1309
10:00-10:30 Dealing with JVM limitations in Apache Cassandra Free Java K.4.401
10:00-10:30 Improving Firefox startup time on Android Mozilla UD2.218A
10:00-10:30 openSUSE on ARM CrossDistribution H.1301
10:00-10:40 Toolkits on Wayland - how we're doing! CrossDesktop H.1308
10:00-10:45 Systems Management with Matahari Configuration and Systems Management K.3.601
10:00-10:45 Anatomy of a role playing game Open Source Game Development AW1.120
10:00-10:45 Introduction to pkgng BSD K.4.201
10:00-10:50 Voice Applications for the Modern Open Source Hacker Network and IO K.1.105
10:00-10:50 CoApp: Packaging Open Source software for Windows System Janson
10:00-10:55 Ganeti: "how you can use it" Virtualization and Cloud Chavanne
10:00-11:00 KMS plane support in Wayland K.3.401
10:00-11:00 eLuaBrain: a 32-bit MCU based educational computer Embedded Lameere
10:00-11:00 reloaded! CrossDistribution H.1302
10:00-11:00 The Next Steps for the Pharo Vision Smalltalk AW1.126
10:00-12:00 OpenSC codesprint Security H.2214
10:05-10:45 Perlude: a taste of Haskell in Perl Perl AW1.121
10:10-10:55 Introduction of the Genode OS Framework Microkernel OS K.3.201