Containers are everywhere. Whether teams decide to use bare containers, orchestrators or enterprise solutions built on top of these technologies, the result is the same in terms of testing.

With rolling updates and fault-tolerant clusters getting more popular every day in organizations of all sizes, availability issues are now easier to handle, whereas the complexity of reliability in our systems stays the same or even increases.

Given the nature of distributed systems, a traditional approach towards testing is not enough. Smoke, integration and unit tests should be only the entry point, while e2e tests are no longer a luxury item in our developer toolset. However, writing and running them with scale and time constraints in mind is a task that is easier said than done.

Since Go was born with tools for developers in mind, it's a great language thatenables productivity, which sometimes is cited as an excuse to avoid writing stronger e2e cases. In this talk, I will share my experience of building a testing framework for distributed systems, revisiting common mistakes, how to avoid them, and I will show how to take advantage of the power of Go to write reliable e2e tests to make sure our systems' behaviors really match our original expectations, with scalability in mind through automation.