Command line tools for containers—using Snyk with Buildah, Podman, and Skopeo
9 de diciembre de 2020
0 minutos de lecturaAs the container ecosystem has matured, the one thing we’re not short on is options—both in terms of the software we use, and how we plug it all together.
One of these options would be the combination of Buildah, Podman, and Skopeo—three open source command line tools with their origins in the RedHat ecosystem. As its name suggests, Buildah provides a wide range of functionality around building OCI compliant containers and images, Podman provides for the management and execution of OCI containers, while Skopeo is a tool for working with container registries.
In this blog post, we’ll look at how the use of open standards allows Snyk’s container scanning to work with all of these technologies.
First of all, let’s look at Skopeo. This is a command line tool for moving container images around between repositories, and can also save images locally. Skopeo supports outputting images either as Docker archives or OCI archives, both of which Snyk supports when scanning from the filesystem. Let’s use it to download an image from Quay.io and scan that locally from our filesystem :
As we can see, this particular image is using an End Of Life version of Ubuntu, and so contains multiple vulnerabilities that will not be fixed. So, using Skopeo, we can download images from any standards-compliant container registry in formats which Snyk supports, and scan them locally.
Buildah
Next, let’s take a look at Buildah. Buildah is a command line tool which builds images, it doesn’t run them, so it’s intended to be used in conjunction with Podman. It’s also designed to operate in rootless mode, utilizing user namespaces in the Linux kernel to segregate a root-like shell which only has user-level privileges. This enables administrators to empower developers to build and maintain container images, whilst at the same time following principles of least privilege within the build environment.
Although Buildah can work with Dockerfiles in a similar way to Docker itself, it also allows you to develop containers and images using other workflows, for example, Bash or Python. This can have advantages in terms of flexibility, and can also make debugging of container builds easier—Buildah allows for things like mounting of the intermediate container during the build process, so scripts can do things like check for correct execution before continuing, build software externally or even take user input. This provides an alternative to the sometimes convoluted multi-stage build processes which have evolved as best practice when using Docker.
Let’s take a look at an example of using Buildah in practice.
One of the first things to note about Buildah is that we can just use Dockerfiles to build our containers, if that is the workflow that works for you. Taking this Dockerfile as an example :
Here we’re using the CentOS 8 image as our base, cloning the GNU Hello source code, installing some prerequisites, and then building and installing the binary.
We can build this automatically using Buildah with the bud ( build-using-dockerfile) switch :
This is clearly not good practice, since we include both the source code and the build prerequisites in our final image, and this image breaks most of the best practices for secure image development. Using Docker, the best practice here would be a multi-stage build. We could use a multi-stage Dockerfile using Buildah, but more interestingly we can also use native Buildah commands, and write scripts to do so. To mirror exactly what we did with the Dockerfile above, we would run the following script :
However, Buildah also allows us to mount the intermediate container during the build process, so we can actually do our source code build on the host, and simply copy the resulting binary into our container before we commit to an image. In this way, we can take advantage of the host resources during our builds, and do other things like use the host package manager to install packages into our container. We also don’t need root access for that, as Buildah supports using user namespaces to allow you to mount without privileges, by using the unshare command :
As you can see in the above example, after running the unshare command we now have a root shell, although this is in a user namespace without full root privileges. This allows us to mount the container filesystem during the build process. Since this is a bash script, we can also potentially leverage all the functionality that a programming language gives us—we could use conditionals, loops, or even request user input during the process.
Podman
Once we run this script and build our image, we can also scan it using Snyk, and this is where Podman comes in. Podman is a daemon-less container engine for developing, managing, and running OCI containers. Since Buildah and Podman are based around open standards, Snyk has always been able to scan images created or pulled by Podman, by using Podman to save the image to disk and scanning it from the filesystem. Podman supports saving images as both a standard Docker archive or in OCI archive format, both of which Snyk supports.
Now, since our base image is a public image which exists in Dockerhub, if we want to scan this image using Snyk CLI, we could also just do :
In this case, Snyk will interact directly with Dockerhub and just scan the upstream image.
However, if the image only exists locally in Podman, or if it’s in a private repository in Dockerhub which requires a login, neither of those cases will work. As an example, let’s try to scan our built image directly from Podman:
For these use cases, Snyk uses the local Docker client, either through the Registry API or by using the Docker socket. Since there is no Docker binary on the system, here Snyk has attempted to use the Registry API and failed to connect. By default, Snyk doesn’t know those images exist locally in Podman, or how to use Podman to interface with upstream.
Luckily, Podman provides us with some easy compatibility features through the podman-docker package. This gives us a fake Docker binary, which is basically just a shell wrapper to the Podman binary, and adds a symlink between the Podman API socket and the default location of the Docker socket file. Out of the box, this package creates a symlink to /var/run/podman/podman.sock, which is the default when running the Podman API as root.
This still doesn’t work, but this time we’ve got a different error. Snyk has detected a binary called docker, and so is now trying to connect to the default location of the privileged Docker socket. Since we are not yet running the Podman API, it can’t connect and so fails again.
Now as we saw earlier, the podman-docker package creates a symlink from /var/run/podman/podman.sock to /var/run/docker.sock, assuming that the Podman API is going to be run privileged.
One of the main issues with the Docker socket is that by default it runs privileged, which is considered a security risk in certain environments. For most local development environments we don’t need to run the Podman API that way, and can run an unprivileged socket somewhere else. We also have a method for communicating that location to Snyk by using the DOCKER_HOST environment variable, which supports both tcp:// and unix:// URL formats.
In a different shell, let’s run the Podman API with an unprivileged socket:
This will run foregrounded, so you will have to ctrl-C the process when finishing the test. Now we need to set the DOCKER_HOST environment variable:
And finally, let’s try and scan our local image again with Snyk :
Success! Snyk is now working correctly and talking to Podman using the unprivileged socket.
To configure DOCKER_HOST permanently in your shell, you can add the export command to your .bashrc file which will persist the setting.
Finally, we’d really like to have this whole setup automatically available to us whenever we log in. To do this we can take advantage of the socket activation facilities in systemd to automatically start our socket when we log in and try to connect to it.
First we need to configure the socket :
And then tie the socket to a service :
Finally, we reload the systemd configuration and enable the service!
Conclusion
So, there we have it—Snyk CLI image scanning with Podman working in exactly the same way as with Docker, allowing developers easy access to comprehensive security scans of local Docker or OCI images as part of their development workflow, without requiring raised privileges. We also saw how both Skopeo and Buildah can be used with Snyk, through the power of open standards!
Don’t already have a Snyk account? Sign up for free and use Snyk to scan both container images and open source dependencies.
Introducción a Capture the flag
Aprende a resolver los desafíos de Capture the flag viendo nuestro taller virtual 101 bajo demanda.