In general, MUST’s correctness checking includes errors that already manifest - segmentation faults or incorrect results - and many errors that are invisible to the developer or only manifest with certain HPC systems and MPI libraries. MUST works by intercepting MPI calls of a target application at runtime, allowing for bookkeeping of the current program state. Thus, MUST can cope with the complex MPI semantics of, e.g., (a) collectives, (b) wildcards, or (c) datatypes.

As a consequence of relying on intercepting MPI calls, though, MUST is unaware of the effective type of the allocated void* buffers used for the low-level MPI API. To that end, TypeART was developed to track memory (de-) allocation relevant to MPI communication. TypeART instruments heap, stack and global variable allocations with a callback to our runtime. The callback consists of (a) the memory address, (b) the type-layout information of the allocation (built-ins, user-defined structs etc.) and (c) number of elements. Thus, with TypeART, MUST can check for type compatibility between the type- less MPI communication buffer and the declared MPI datatype. Our tools also handle derived datatypes with complex underlying C/C++ data structures.