Software is eating the world, and networking is no exception. Gone are the days when there was a distinct boundary between the application servers and the networking connecting them together. In the cloud native era that we live in, much of traditional networking between applications ends up being defined entirely within the Linux kernel networking stack, sometimes enhanced by eBPF programs, to provide highly scalable and efficient connectivity between Linux containers. When network packets go missing, its no longer sufficient to capture packets with tcpdump, or trace the path through the Linux firewall rules. You may have to look deeper into the Linux kernel's networking subsystem and trace the packet flow through each of the kernel networking functions watching for packet filtering or modifications.

Cilium's Packet Where aRe You (PWRU) was purpose built to help discover problems deep in the Linux kernel's network subsystem. It is a great diagnostic tool to reach for when you've ruled out DNS or an obvious firewall rule blunder. In this talk I'll review how PWRU works, and walk through a few usage examples.