Microservices are becoming very popular recently. But how to run automated end-to-end tests against your product when your architecture and deployment processes are complex? How to orchestrate your test environment when your product is a SaaS application but can also be installed on-premises on bare metal or deployed into the cloud on to your Kubernetes cluster?

Using containers to automatically test your application is a great use case, but sometimes running tests only against containers might not be enough.

We decided to create a click-driven, black-box integration testing framework, using Ruby, headless Chrome, Docker and Kubernetes, to test GitLab.

This talk is a story about building this integration testing framework. It is about obstacles we had to overcome, essential complexity that we had to embrace, but also about problems that we still need to solve and our hopes for the future.