HPC computing environment and especially super computing has its bag of challenges when it comes to software deployment. As scientists we often do not get root access which means that we either depend on what software is available or we build software in a dedicated directory using tools such as Brew, Conda or even from source. Unfortunately these solutions depend on already installed tools from an underlying distribution, often proprietary or dated compilers, and, for example, modules. Any binary that gets produced, therefore, tends to be totally unique, both in the generated binary and its set of dependencies. This is bad, both for trouble shooting and for pursuing reproducible science. With GNU Guix we are developing a fully reproducible free software stack including all dependencies targeting cross-builds for heterogeneous computing environments, such as the Beacon Intel PHI-based supercomputer, without having a requirement for containers. In this talk I will describe the challenges we are overcoming and what it will mean for the HPC community to have architecture optimized and fully reproducible build targets that can be deployed in a non-root, non-container environment.