Most of the day to day tests in LLVM are regression tests executed by Lit, structured as source code or IR to be passed to some binary, rather than test code directly calling the code to be tested. This has many advantages but can make it difficult to predict which code path is executed when the compiler is invoked with a certain test input, especially for edge cases where error handling is involved. The goal of this work was to help developers create good test coverage for their patch and enable reviewers to verify that they have done so. To accomplish this we have introduced a tool that can be fed a patch as input, add coverage instrumentation for the affected source files, runs Lit tests, and records which test cases cause each counter to be executed. For each counter we then report the test cases executing the counter, but perhaps more importantly we also report the test cases executing the counter that are also changed in some way by the patch, since a modified line that results in zero counter isn’t properly tested, unless it’s intended to be a non-functional change.

In this presentation, we will understand what are the technologies involved, the shortcomings to implement the solution, the idea of implementation, and the final demo result by using a llvm-project patch.