Brussels / 3 & 4 February 2018


Geographically distributed Swift clusters

The OpenStack Swift object storage service achieves high availability and durability by replicating object data across multiple object servers. If one disk or object-server fails then its data is always available on other object servers. Swift is able to apply this principle to not just disks and object servers but also independent availability zones and regions. This enables the deployment of so-called Global Clusters which provide a single object namespace spanning multiple geographically dispersed data-centres, each offering independent local access to object data.

This talk will describe some of the mechanisms that Swift provides for configuring and optimising Global Clusters. We will briefly describe how Swift's consistent hashing Ring maps objects to object servers and how that mapping algorithm can be configured to distribute copies of each object across data-centres. We will show how Swift's read and write affinity settings can be used to optimise WAN traffic in a Global Cluster. Finally we will discuss some of the challenges we faced when implementing Global Cluster support for erasure coded objects, and how those were overcome by enhancements to the Ring and the erasure coding write path.

Talk outline:

  • Why global clusters?
  • the Ring - regions, zones
  • replica placement - trade off between dispersion and balance
  • read/write affinity options - when (not) to use them
  • new feature: per-policy affinity settings
  • illustrate with typical global replication policy

  • Global erasure coding (EC)

    • overview of Swift EC support
    • briefly cover what EC is with an example policy
    • Describe problem statement with EC and global cluster
    • Show how EC lib performance degrades as parity count increases
    • Describe solution: duplicated fragment scheme Composite Rings
  • Future work

    • optimise the search for an EC fragment set to avoid fetching duplicates
    • make reconstructor rebuild more efficient by becoming region aware


Alistair Coles