Online / 6 & 7 February 2021


buildtest: HPC Testing Framework for Acceptance Testing

Buildtest ( is an HPC testing framework to aid HPC facilities to perform acceptance testing for their system. HPC systems are growing in complexity, with a tightly coupled software and system stack that requires a degree of automation and continuous testing. In the past decade, two build frameworks (Spack, EasyBuild) have emerged and widely used in HPC community for automating build & installation process for scientific software. On the contrary, testing frameworks for HPC systems are limited to a few handful (ReFrame, Pavilion2, buildtest) that are in active development.

In buildtest, users will write test recipes in YAML called buildspecs that buildtest process to generate a shell script. buildtest utilizes versioned-based JSON Schema for validating buildspecs and currently, we support two main schemas (compiler, script). The script schema and compiler schema are used for writing traditional shell-scripts (bash, sh, csh), python-scripts and single source compilation test. In this talk we will present an overview of buildtest and how one can write buildspecs. Furthermore, we will discuss Cori Testsuite ( in buildtest with several real examples on testing various components for Cori system at NERSC.


Photo of Shahzeb Siddiqui Shahzeb Siddiqui