The RUM Archive is the Real User Monitoring-focused counterpart to the synthetic results from the HTTP Archive. It contains (aggregated and anonimized) measurements from millions of end users, as seen by the mPulse RUM tool on the Akamai CDN network. Containing over 1.5 years of data, the RUM Archive doesn't just provide insight into current real-world performance aspects, but also help us to understand trends and evolutions over time, for a variety of metrics and data dimensions.

In this talk, we will first discuss how the RUM Archive works underneath; the data it holds and some examples on how to extract it yourself. We then move to analyzing some of the data contained within, looking at differences between for example browsers, operating systems and devices, and how those can impact performance metrics. We also discuss RUM-specific challenges, such as dealing with noise in the measurements, and show how this can make a big difference in the conclusions we draw from the data.

We end with some critical discussion on the state of Web performance testing, emphasizing the need for both synthetic and RUM measurements, and taking a long hard look at our over-reliance on CrUX, the Google Core Web Vitals and Chromium.