This talks will present how to perform benchmarking of Big Data systems, from low-powered devices, HPC clusters, to cloud IaaS and PaaS. It will guide participants on how to define clusters, select benchmark suites and configuration with the ALOJA open source tools. ALOJA (http://aloja.bsc.es) is a research initiative from the Barcelona Supercomputing Center to explore new cost-effective hardware architectures for Big Data. During its first year, ALOJA's benchmarking efforts have produced the largest public repository with over 50,000 Hadoop benchmark runs. The searchable repository features different applications for Hadoop, software configurations, data sizes, and more than 100 different hardware deployment options.