FOSDEM 2020
/
Schedule
/
Events
/
Developer rooms
/
HPC, Big Data, and Data Science
/
Build for your microarchitecture: experiences with Spack and archspec

Build for your microarchitecture: experiences with Spack and archspec

Track: HPC, Big Data, and Data Science devroom
Room: UB5.132
Day: Sunday
Start: 15:00
End: 15:25

In HPC, software is typically distributed as source code, so that users can build optimized software that takes advantage of specific microarchitectures and other hardware. While this approach provides a lot of flexibility, building software from source remains a huge barrier for users accustomed to simple, fast binary package mangers. Most package managers and container registries label binaries with a high-level architecture family name, e.g., x86_64 or ppc64le, but there is no standard way to label binaries for specific microarchitectures (haswell, skylake, power9, zen2, etc.).

We’ll present a new project called “archspec” that aims to bridge this gap. Archspec provides a standard set of human-understandable labels for many popular microarchitectures. It models compatibility relationships between microarchitectures, and it aggregates information on ISA extensions, compiler support, and compiler flags needed to optimize these machines. Finally, it provides a standard set of names for both microarchitectures and ISA features. These features allow container tools and package managers to detect, build, and use optimized binaries.

Archspec grew out of the Spack package manager, but it is intended for widespread use by other build, packaging, and containerization tools. We will describe how it has been used in practice so far, how it has simplified writing generic packages, and our plans to get contributions from vendors and the broader community.

Expected prior knowledge / intended audience: Audience should have basic knowledge of build systems, as well as some knowledge about processor architectures. There will be some brief background on this in the talk. This will be interesting to HPC users, developers, packagers, and admins, as well as to anyone writing tools that deal with microarchitecture metadata (like container systems).

Speaker bio: Todd Gamblin is a Senior Principal Member of Technical Staff in the Advanced Technology Office in Livermore Computing at Lawrence Livermore National Laboratory. His research focuses on scalable tools for measuring, analyzing, and visualizing parallel performance data. In addition to his research, Todd leads LLNL's DevRAMP (Reproducibility, Analysis, Monitoring, and Performance) team and the Software Packaging Technologies project in the U.S. Exascale Computing Project. He created Spack, a popular open source HPC package management tool with a community of over 450 contributors. Todd has been at LLNL since 2008.

Links to code / slides / material for the talk (optional): To be provided closer to FOSDEM.

Links to previous talks by the speaker: https://www.youtube.com/watch?v=DRuyPDdNr0M https://www.youtube.com/watch?v=edpgwyOD79E&t=2891s https://www.youtube.com/watch?v=BxNOxHu6FAI https://insidehpc.com/2019/03/spack-a-package-manager-for-hpc/ https://www.youtube.com/watch?v=iTLBkpHskzA

See https://tgamblin.github.io/cv/todd-cv.pdf for more (including tutorials and other presentations at major conferences)