The VCL programs used to configure the behavior of our edge CDN come with multiple test cases. The environment needed to run the Varnish Test Cases (VTC) can currently be reproduced by using Vagrant, see
./modules/varnish/files/tests/Vagrantfile and the instructions for running the tests.
Although the current approach based on Vagrant does work, there is actually no need for a full-fledged VM to run the tests, and a container-based approach would suffice. Other than making things faster and arguably easier, using containers (Docker, I suppose?) would help pushing T128188 forward.
In order to run VTC tests, the following are needed:
- varnish
- one or more VCL files
- one or more VTC files
Running the tests is made a little convoluted by the fact that the VCL files are actually ERB templates that need to be rendered by Puppet. For this reason the test infrastructure comes with a Python script and a shell wrapper companion responsible for running the Puppet Catalog Compiler against a given changeset (eg: 702896) and node (eg: cp3050.esams.wmnet). The container-based approach needs to take this into account, and ensure that the scripts can do their work. This includes all the relevant network communication with both puppet-compiler.wmflabs.org and config-master.wikimedia.org. The former is needed for the aforementioned catalog compilation, while the latter is used to figure out the cache cluster a host belongs to (ie: text vs upload).
This task is about porting the test infrastructure from Vagrant to Docker. We will need to produce:
- A Dockerfile equivalent to the current Vagrantfile for building the Docker image
- A script for running the tests given a Puppet changeset and hostname. Updating the current shell script may be a good idea, there is no need to keep the current Vagrant approach once we have a stable alternative