RelEng has landed on buildkitd as the best option for building production bound images from protected GitLab CI workloads. (See T307599: Investigate alternatives to docker-in-docker for container image creation in GitLab and T307810: Investigate buildkitd instances as image builders for GitLab.)
Per discussion with @Jelto and the rest of RelEng, it sounds like there are security concerns with integrating any third party k8s cluster with our trusted GitLab runners, so instead of a k8s deployment, we will target running buildkitd in rootless mode via dockerd on those the trusted runners using a WMF packaged image from docker-registry.wikimedia.org.