FOSDEM is the biggest free and non-commercial event organized by and for the community. Its goal is to provide Free and Open Source developers a place to meet. No registration necessary.

Anil Madhavapeddy
Day Sunday
Room K.1.105
Capacity 809
Start time 12:00
End time 12:50
Duration 00:50
Track Network and IO Track

The Wild West of UNIX I/O

and how we might tame it

The Unpredictability of UNIX I/O (and how we might tame it)

Inter-process communication and remote procedure call facilities have existed in operating systems for many decades. Ever since the first parallel applications ran on time-sharing machines, programmers have sought ways to communicate between processes running on a single machine, and the first networked applications introduced the concept of sending a message to trigger a remote action.

And today, these primitives are more relevant than ever before: parallel programming on clusters of machines relies heavily on facilities to pass data between processes and hosts. On a higher level, data-flow frameworks for parallel processing of large data sets (such as Hadoop or CIEL) depend on passing data between different tasks, which may run anywhere, including local to a machine, on a networked cluster, or far away in a virtualised wide-area "cloud".

And yet, we are stuck with UNIX communication APIs closely coupled to the underlying mechanisms used to implement them: the programmer choice of sockets, pipes or shared memory constitutes an implicit choice of a whole set of assumptions about the relative locations of the communicating parties, as well as how the message is to be delivered. Worse even, the implicit trade-offs may not be the same in a different environment, and thus the programmer's choice of API depends on assumptions about the runtime environment (hardware, software and setup) in addition to the characteristics inherent to a mechanism implied.

This talk will firstly discuss the impossibility of using current APIs efficiently (via benchmarks on a diverse set of hardware (from many-core AMDs to the experimental Intel SCC). Finally, I will describe our work on introducing a hierarchical name system and extended socket API that adds support for automatic transport selection and reconfigurable sockets. This permits many NUMA-related optimisations on single hosts, for VMs to switch to shared memory communication if on the same physical host, and for seamless network-wide protocol upgrades to multi path TCP or TCPcrypt.

Concurrent events:

When Event Track Where
10:30-12:15 LPI Exam Session 3 Certification Guillissen
11:00-17:00 LPI Item Writing Workshop Certification UA2.114
11:30-12:30 Working with contributor communities (round table) CrossDistribution H.1301
11:40-12:30 and Qt 5 CrossDesktop H.1308
11:45-12:30 Integrating LLVM into FreeBSD BSD K.4.201
11:45-12:30 The Dynamic Data Driven Worlds of WorldForge Open Source Game Development AW1.120
11:55-12:15 The LemonLDAP::NG Project Perl AW1.121
12:00-12:05 OpenPGP keysigning Certification Ferrer
12:00-12:25 Cluster internals MySQL and Friends H.1309
12:00-12:25 VDSM - The oVirt Node Management Agent Virtualization and Cloud Chavanne
12:00-12:30 How Eclipse Works Free Java K.4.401
12:00-12:30 An introduction to jQuery Mobile Smalltalk AW1.126
12:00-12:30 Continuous integration with Mozmill for Firefox Mozilla UD2.218A
12:00-12:35 Works with persistent graphs using OrientDB Graph Processing AW1.125
12:00-12:50 coreboot - The last frontier: Laptops System Janson
12:00-13:00 Safe upgrade of embedded systems Embedded Lameere
12:00-13:00 Colour Management in Compositors. K.3.401
12:05-14:00 CAcert Assurance Party CAcert Ferrer
12:15-13:00 29,000 packages in 24 hours - Releasing Debian CrossDistribution H.1302
12:25-12:45 LedgerSMB: Open source accounting running on Perl Perl AW1.121
12:30-12:55 Optimising SQL applications by using client side tools MySQL and Friends H.1309
12:30-13:00 RoarVM, Sly Smalltalk AW1.126
12:30-13:00 Howto: Extensions for Thunderbird Mozilla UD2.218A
12:30-13:00 OpenJDK on ARM: Quo vadis? Free Java K.4.401
12:35-13:15 Session Border Control with SEMS Telephony and Communications H.2213
12:40-13:25 Pandora - a mobile Linux computer including gaming controls Open Source Game Development AW1.120
12:45-13:15 Graphity: an efficient neo4j based graph model for retrieving the top k news feeds for users in social networks Graph Processing AW1.125