The Wine 'conformance tests' are the means by which Wine developers explore and document the behavior of the Windows API. The trouble is they typically have access to at most one Windows version making it impossible to distinguish between quirks in that version and behavior Windows applications may rely on.

The WineTestBot is meant to not only solve that issue but also act as a filter to weed out bad patches before they get committed.

It was initially developed as a VMware-based Windows test farm to solve that issue. It was then extended to test all conformance test patches to weed out the bad ones before they get committed. Now the challenges are to migrate it to a fully open-source architecture; extend it to validate all Wine patches and thus to non-Windows platforms; and to also run the tests on real hardware for the 3D and sound tests.

This talk will first provide a quick refresher on the conformance tests. Then it will present the architecture of the WineTestBot, how it compares to other testing frameworks, the implications of extending it to all Wine patches, and to testing on real hardware.