Codefresh + Snyk = ship fast and securely

Modern software development is about writing code. Not building, not shipping, but developing — we code, we merge, it builds, it ships. It’s important to test on both sides of the repo frontier, between the code and the automated part. The goal is to test before we merge so the changes go through the pipeline smoothly.

Pipelines are not meant for us developers to test our changes for the first time, it's meant to gate if our code does not satisfy the criteria for shipping the new release. It’s like a final checklist while the rocket is still on the launchpad. In an ideal world, it's even a bit too late if we need to fix things during a pipeline. We’d rather do that during development and/or merging. However, while the world is not always ideal, we should still strive to keep changes in the development phase, as a general rule of thumb.

To achieve this, test earlier to provide feedback as far left as you can, such as on your machine or in your code repository. Of course, you’ll still need to build this pipeline in the first place, integrating the necessary tests. This is where Codefresh comes in. It allows you to build automated pipelines to ship out your apps in containers and Kubernetes deployments.

These tests MUST include security. It's simply a must. If you’re not convinced, stop reading this post and go talk to your security team/mate/peers/meetup group or even us. We’ll each explain why you should care... this is where Snyk comes in.

We are going to take 5-10 minutes and build this pipeline in Codefresh, baking in security testing with Snyk. The testing will be short and sweet so we can go back to coding and ultimately shipping features. Both Codefresh and Snyk have free plans, so you can follow along at no cost. I also gave a webinar session with the folks from Codefresh, so if you prefer to watch the video, you can check it out here.

We are going to build a sample Node app and ship it into a Docker container that we'll push on to Docker Hub. From there, another pipeline will kick in and deploy the output as a Kubernetes application. Codefresh will pick up on changes merged into our repo, and build the Docker images, then run the Snyk tests to determine if it’s safe to proceed shipping the resultant output.

First, let’s have a quick primer on applications and containers. The anatomy of a typical application running on a container infrastructure is illustrated below.

The pipeline will get my application source from my GitHub repo, build it into a Docker container, then test both application dependencies as well as OS dependencies for security issues before pushing it to Docker Hub. Assuming we are vulnerability free, it will not break.

Let's now create this pipeline for real!

1. Sign up for new Codefresh and Snyk.io accounts.

2. Add a repo in Codefresh.

For this demo, we'll use the https://github.com/snyk-playground/codefresh-pipeline-snyk-app-docker-scan repository.

3. Next, select Start from template as your build method.

We’re using a NodeJS application here. Let’s select that and click next.

We’ll use the stock template for now. Feel free to tweak based on your needs.

Alright, our new repo was added, let’s now proceed to defining the pipeline.

4. Now, build the repository by clicking Create pipeline.

Let’s first define some required environment variables. We’ll make sure to encrypt any secrets. Add a new variable called ‘SNYK_TOKEN’.

The value can be found on https://app.snyk.io/account.

While you’re there, add another variable called SNYK_ORG with the value as the end part of your URL https://app.snyk.io/org/. In my case, it’s aarlaud-snyk-demo

That’s all for Snyk.

Now, we’ll need a few additional env vars to pull the image down from the Codefresh registry and also to push the final image to Docker Hub.

For Codefresh to access our private image registry, add the following variables (see for more details here https://codefresh.io/docs/docs/docker-registries/codefresh-registry/#generate-cfcr-login-token):

• CF_USER_NAME - my Codefresh username (eg.aarlaud for me)

• CFCR_ACCOUNT - for me, it’s the same as my username aarlaud

• CFCR_LOGIN_TOKEN - generate a token on https://g.codefresh.io/user/settings, and encrypt it!

For the Docker Hub integration, we’ll store the image name that we want to push our final Docker image into the Hub under:

IMAGE_NAME - for example in my case aarlaudsnyk/trainingapp

Your set of variables should look similar to this:

Lastly, we’ll need the Docker configuration setup in Codefresh Integrations section:

It does require an account on Docker Hub (https://hub.docker.com/), so create one if you have not already done so and set up the Docker registry integration:

Alright, now we are ready to proceed!If you take a look at your triggers, you’ll see that every commit will trigger a build. Note that because we created this pipeline off the master branch, only commits in master will trigger the build. This is good enough for now, and easy to change or duplicate for other branches for other environments.

If we look at the workflow, we will recognize the Docker template that we saw earlier. It might at some point be better to move this into a dockerfile.

Now, let’s try to build this to see if it works as expected before going any further.

You’ll see the build starting:

And now my app has built.

Now, Codefresh automatically puts this image into your own private Docker registry on your Codefresh account.Looking at the Images section, I can see my freshly built Docker image:

Additionally, before simply pushing this image to Docker Hub, I want to ensure that we are free of security vulnerabilities.

5. Going back to the pipeline, let’s add a snyk test command to test the dependencies for any security vulnerabilities.

We’ll first install Snyk and then run it using very simple commands aimed at breaking if we have any high severity issues. I don’t want to break for low or medium so much until I get a better grip on my workflow.

> npm install -g snyk
> snyk test --severity-threshold=high

Hit save, then build it again to run it. You’ll see an extract step in the pipeline for Running Unit Tests. It is probably going to fail because of some issues. We’ll fix them later. Let’s now scan the Docker image itself for the OS component of my app.

When Codefresh builds the app, it pushes the image to our private registry. It then pulls it to run the unit tests. We’ll repeat that process to run a second snyk test targeted at the Docker image.

To do that, we’ll use a Codefresh composition. Look at the docs for more details, but we’ll switch to yaml to give us more capabilities.

Now in yaml mode, let’s first add these 3 lines right after the “version: ‘1.0’”stages:

-scan
-promote

This change is purely cosmetic – it will create a bucket-like section in the UI.Let’s tweak this a bit for our existing RunningUnitTests section to use the SnykAppScan section. Note that besides the title changes, we’ve added a stage section to put this in the right UI bucket, and we are passing more directly the environment variables, cleaning up a few things.

SnykAppScan:
title: Snyk Test Application Dependencies
stage: scan
image: '${{BuildingDockerImage}}'
working_directory: IMAGE_WORK_DIR
environment:
- SNYK_TOKEN=${{SNYK_TOKEN}}
- SNYK_ORG=${{SNYK_ORG}}
commands:
- npm install -g snyk
- snyk test --severity-threshold=high
on_success:
metadata:
set:
- '${{BuildingDockerImage.imageId}}':
- CF_QUALITY: true
on_fail:
metadata:
set:
- '${{BuildingDockerImage.imageId}}':
- CF_QUALITY: false

Now, let’s add a SnykScanImage section, this time for our Docker image scanning portion. Stage is also scan here, so it’s in the same bucket as our app dependencies scanning stage. The composition docs mentioned allow you to run multiple services to run your tests.

In our case, BuildingDockerImage will be the image output of the build stage. We’ll want to test that image from the outside, so we are composing our test with a scan service running another image I previously built, aarlaudsnyk/snyk-container-scan-docker. This image will simply pull together the required pieces for Snyk to perform a test on a Docker image. The source can be seen here:

SnykScanImage:
title: Snyk Test Docker OS Dependencies
stage: scan
type: composition
composition:
version: '2'
services:
targetimage:
image: ${{BuildingDockerImage}} # Must be the Docker build step name
command: sh -c "exit 0"
labels:
build.image.id: ${{CF_BUILD_ID}} # Provides a lookup for the composition
composition_candidates:
scan_service:
image: aarlaudsnyk/snyk-container-scan-docker
command: python snyk-cli.py "${{IMAGE_NAME}}:${{CF_BRANCH_TAG_NORMALIZED}}"
environment:
- SNYK_TOKEN=${{SNYK_TOKEN}}
- SNYK_ORG=${{SNYK_ORG}}
- CFCR_ACCOUNT=${{CFCR_ACCOUNT}}
- CF_USER_NAME=${{CF_USER_NAME}}
- CFCR_LOGIN_TOKEN=${{CFCR_LOGIN_TOKEN}}
depends_on:
- targetimage
volumes: # Volumes required to run DIND
- /var/run/docker.sock:/var/run/docker.sock
- /var/lib/docker:/var/lib/docker
add_flow_volume_to_composition: true
on_success: # Execute only once the step succeeded
metadata: # Declare the metadata attribute
set: # Specify the set operation
- ${{BuildingDockerImage.imageId}}: # Select any number of target images
- SECURITY_SCAN: true

on_fail: # Execute only once the step failed
metadata: # Declare the metadata attribute
set: # Specify the set operation
- ${{BuildingDockerImage.imageId}}: # Select any number of target images
- SECURITY_SCAN: false

The section of interest from the yaml file above is:

image: aarlaudsnyk/snyk-container-scan-docker
command: python snyk-cli.py "${{IMAGE_NAME}}:${{CF_BRANCH_TAG_NORMALIZED}}"

We use that image to then run the Python command which primarily does 2 things:

• Pulls down the image to test locally

• Runs ‘snyk test --docker ’

This image is crafted to pull the image from Codefresh’s registry if the following env vars are defined

• CFCR_ACCOUNT

• CF_USER_NAME

• CFCR_LOGIN_TOKEN

This allows us to test the freshly built image. Otherwise it’ll default to attempt to pull down an image from Docker Hub instead.

Lastly, let’s add the Docker Hub registry push at the end, to actually push this image if we are passing our tests successfully:

PushingToDockerRegistry:
title: Pushing to Docker Registry
stage: promote
type: push
candidate: '${{BuildingDockerImage}}'
tag: '${{CF_BRANCH_TAG_NORMALIZED}}'
registry: aarlaudsnyk

Nothing crazy here, Codefresh makes it a breeze to push into Docker Hub as long as the integration has been configured (see a bit earlier).Note the stage being promote to distinguish it from the scan tests.

All together, it should look like something like this

version: '1.0'
stages:
- scan
- promote
steps:
BuildingDockerImage:
title: Building Docker Image
type: build
image_name: ${{IMAGE_NAME}}
working_directory: ./
tag: '${{CF_BRANCH_TAG_NORMALIZED}}'
dockerfile:
content: |-
FROM node:8.0-alpine AS builder
WORKDIR /app
COPY package.json /app
# Creating tar of productions dependencies
RUN npm install --production && cp -rp ./node_modules /tmp/node_modules
# Installing all dependencies
RUN npm install
# Copying application code
COPY . /app
SnykAppScan:
title: Snyk Test Application Dependencies
stage: scan
image: '${{BuildingDockerImage}}'
working_directory: IMAGE_WORK_DIR
environment:
- SNYK_TOKEN=${{SNYK_TOKEN}}
- SNYK_ORG=${{SNYK_ORG}}
commands:
- npm install -g snyk
- snyk test --severity-threshold=high
on_success:
metadata:
set:
- '${{BuildingDockerImage.imageId}}':
- CF_QUALITY: true
on_fail:
metadata:
set:
- '${{BuildingDockerImage.imageId}}':
- CF_QUALITY: false

SnykScanImage:
title: Snyk Test Docker OS Dependencies
stage: scan
type: composition
composition:
version: '2'
services:
targetimage:
image: ${{BuildingDockerImage}} # Must be the Docker build step name
command: sh -c "exit 0"
labels:
build.image.id: ${{CF_BUILD_ID}} # Provides a lookup for the composition
composition_candidates:
scan_service:
image: aarlaudsnyk/snyk-container-scan-docker
command: python snyk-cli.py "${{IMAGE_NAME}}:${{CF_BRANCH_TAG_NORMALIZED}}"
environment:
- SNYK_TOKEN=${{SNYK_TOKEN}}
- SNYK_ORG=${{SNYK_ORG}}
- CFCR_ACCOUNT=${{CFCR_ACCOUNT}}
- CF_USER_NAME=${{CF_USER_NAME}}
- CFCR_LOGIN_TOKEN=${{CFCR_LOGIN_TOKEN}}
depends_on:
- targetimage
volumes: # Volumes required to run DIND
- /var/run/docker.sock:/var/run/docker.sock
- /var/lib/docker:/var/lib/docker
add_flow_volume_to_composition: true
on_success: # Execute only once the step succeeded
metadata: # Declare the metadata attribute
set: # Specify the set operation
- ${{BuildingDockerImage.imageId}}: # Select any number of target images
- SECURITY_SCAN: true

on_fail: # Execute only once the step failed
metadata: # Declare the metadata attribute
set: # Specify the set operation
- ${{BuildingDockerImage.imageId}}: # Select any number of target images
- SECURITY_SCAN: false

PushingToDockerRegistry:
title: Pushing to Docker Registry
stage: promote
type: push
candidate: '${{BuildingDockerImage}}'
tag: '${{CF_BRANCH_TAG_NORMALIZED}}'
registry: aarlaudsnyk

6. Let's try building now, so we can get a baseline. We can see our organization looks pretty clean:

After a moment, we get this

Uh oh, looks like we have some issues in our app — it safely stopped the pipeline before shipping something with security issues in it. Let's fix that by following the remediation steps indicated in the output. Bumping the qs package to version 6.0.4 in the package.json will do the trick.

Let's run this again.

We cleared the qs vulnerability, yay! But we now we have something going on with the OS dependencies in our Docker image.

The remediation indication shows that bumping the image should be a good way to go (check out snyk.io/docs in Docker section). So let's bump to node:10-alpine and try this again.

Save => Build => Doing push-ups while building!

Sweet! We now have a fresh baseline and a fully functioning pipeline shipping our app+container to Docker Hub.

We can now go back to coding. Let's make sure we run snyk test for our changes before we merge and/or test our changes using Snyk’s GitHub PR integration. From there the changes will be shipped automatically. Extra bonus, latest addition!

Use snyk monitor to keep an eye on your dependencies over time. Nobody likes babysitting projects, so by running snyk monitor, Snyk will automatically notify us if a new vulnerability has been disclosed for any of the dependencies in the project or Docker image. We just need to add snyk monitor after the snyk test section.

You’ll then see the url of the snapshot in the output for monitoring:

The Docker portion is going to be done automatically if the snyk test passes successfully (meaning there are no known issues). Monitoring snapshot links are also provided.

You can also see the monitored projects on Snyk UI in your Snyk organization:

See the Codefresh pipeline by clicking the badge in the repo (https://github.com/snyk-playground/codefresh-pipeline-snyk-app-docker-scan).

That’s all folks! Stay Secure!