Brussels / 4 & 5 February 2023


A Study of Fine-Grain Compartment Interface Vulnerabilities: What, Why, and What We Should Do About Them

Software compartmentalization decomposes applications into lesser-privileged components that only have access to what they need to do their job. Properly applied, compartmentalization can limit the impact of many memory safety issues by containing corruption within the vulnerable component. Use-cases are plentiful: library sandboxing, protection of SSL keys, sandboxing of network-facing code, and more.

In the last decade we have seen the appearance of many new mechanisms that enable compartmentalization at a relatively low performance cost (Intel PKU, the upcoming Intel PKS, CHERI hardware capabilities, vmfunc). This generated a lot of research with a strong focus on compartmentalizing existing software, at a fine grain (isolating libraries or components), and as automatically as possible. The promises are great: the compartmentalization of legacy software, with a low engineering effort, and at a low performance cost.

Alas, in this process, the interfaces between compartments are often neglected: they are hard to reason about and difficult to secure automatically, and compartmentalizing at finer and finer-grain exacerbates the issue. This is a major problem, as weak interfaces enable for many attacks well-known in the confidential computing world.

In this talk I will present the result of a study on the impact of neglecting compartment interfaces. I will define and classify compartment interface vulnerabilities, and present a fuzzer specialized to catch them. Having applied it to 25 popular applications and 36 possible compartment APIs, revealing 629 interface vulnerabilities, I will present insights into what makes interfaces vulnerable, and how to make them more resilient when compartmentalizing.


Photo of Hugo Lefeuvre Hugo Lefeuvre