Brussels / 2 & 3 February 2019


Storing "Lots Of Small Files" in a Swift cluster

OpenStack Swift is the massively scalable object storage of OpenStack. It can store billions of objects in a single cluster without any disturbances. But what happens if all of your objects are "small objects"?

Because OpenStack Swift stores objects as files on XFS, disks will end-up storing millions of small files, creating large performance and stability issues. This is all related to the constraints of POSIX filesystems.

During this presentation, we will give you an overview of these issues, and explain their root causes. You will then discover what we tried to overcome them, as well as the chosen solution. Eventually, we will present you results of our production deployment.