Automating vulnerability monitoring with Snyk, Prometheus and Grafana

This article is part 3 of Snyk API Wednesdays - our new blog series covering different Snyk API use cases and customer stories. Part 1 covered the Snyk-Watcher by Twilio, and part 2 covered a new integration with Opsgenie using custom webhooks.

Monitoring the vulnerabilities identified across your different applications plays a key role in your organization’s vulnerability management lifecycle - the process of continuously identifying, prioritizing and remediating potential security risks, helping you track your security posture over time and gauge the success or failure of existing programs.

Snyk’s Reporting was designed for exactly this purpose, providing a view of the different security vulnerabilities and license issues found in your different projects. In some cases, though, this might not be enough. Some teams, for example, would rather view this data within their existing monitoring tools which they are accustomed to using. That’s where the Snyk API comes in handy!

Introducing the Snyk Exporter

The Platform team at Lunar strongly believes in empowering developers with the responsibility for addressing security issues as they come up. For this reason, they set out to make it easy for developers to keep tabs on vulnerabilities in their projects using the monitoring tools they were already using - Prometheus and Grafana. The result — the Snyk Exporter — a Prometheus exporter for Snyk.

Written in Go, the exporter enables you to automatically export Snyk’s security testing data into Prometheus by scraping Snyk’s API. You can then view this data within Grafana for monitoring.

How does it work? Once run, the exporter starts a Go process that scrapes the Snyk API at a defined interval that can be configured as needed. In accordance with the Prometheus data model, the API results are then aggregated using a snyk_vulnerabiilities_total metric with labels such as project, severity, and issue_title.

The easiest way to run the exporter is using the available container image.  There are various options you can configure when running the container, such as filtering API results by organization or defining the log level for the exporter.

Note: The exporter requires a Snyk API token, provided in Snyk’s paid plans only.

A few words on Prometheus and Grafana. It would not be an exaggeration to say that these two tools are the de-facto standard for cloud native monitoring. Prometheus is an open source, metrics-based monitoring system while Grafana is an open source visualization tool that sits on top of various data sources, Prometheus included. Together, they provide both the backend and frontend needed for monitoring and troubleshooting the performance of distributed systems.

Let’s take a closer look at how to set up the exporter!

Setting up your vulnerability monitoring stack

If you haven’t already got Prometheus and Grafana set up, here are a few steps to help you quickly get up and running with a local test environment.

There are various ways of installing the two tools, I chose to use Homebrew for the purposes of this demo. This is clearly not a production setup, so refer to the Prometheus docs for more information.

Setting up Prometheus

1brew install prometheus

Before you start Prometheus, you will need to ensure it will scrape the endpoint the Snyk Exporter publishes metrics to - port 9532 by default.

Open the prometheus.yml file, and add the endpoint as follows:

1global:
2  scrape_interval: 15s
3  scrape_configs:
4    - job_name: "prometheus"
5      static_configs:
6        - targets: ["localhost:9090"]
7        - targets: ["localhost:9532"]

You can now start Prometheus with:

1brew services start prometheus

Setting up Grafana

1brew install grafana
2brew services start grafana

Open your browser at http://localhost:3000, and use admin/admin as your credentials. Grafana will then prompt you to enter a new password, after which you will be displayed with Grafana’s welcoming Home page:

The next step is to add your first datasource.

As mentioned, Grafana can pull metrics from various types of datasources, Prometheus being one of them, and also the option highlighted once you hit the Add your first data source button and start the process.

On the setting page that opens up, and for the simplest setup, use http://localhost:9090 as the Prometheus server URL:

Click Save and test at the bottom of the page. You are good to go!

Deploying the Snyk Exporter

Our next step is to run the Snyk Exporter. Before we do, we need to access the Snyk UI to retrieve two details:

1. The Snyk API token for accessing the Snyk API - available at the top of the Account Settings page:

2. An Organization ID - this is optional but also useful for filtering API results by organization. The ID is available on the relevant organization’s general settings page:

With these two details in hand, run the Snyk Exporter container. Be sure to expose port 9532, and enter the Snyk API token and organization ID as follows:

1docker run -p9532:9532 quay.io/lunarway/snyk_exporter --snyk.api-token [SnykAPIToken] --log.level="info" --snyk.organization=[SnykOrganizationID]

The Snyk Exporter starts and begins scraping the Snyk API for the organization specified:

1time="2021-01-29T12:03:03Z" level=info msg="Starting Snyk exporter for organization 'b1403b16-c5cf-4a92-b1d3-28a7868b681e'" source="main.go:65"
2time="2021-01-29T12:03:03Z" level=info msg="Listening on :9532" source="main.go:109"
3time="2021-01-29T12:03:03Z" level=info msg="Snyk API scraper starting" source="main.go:138"
4time="2021-01-29T12:03:07Z" level=info msg="Running Snyk API scraper for organizations: AlphaProduct" source="main.go:172"
5time="2021-01-29T12:03:07Z" level=info msg="Collecting for organization 'AlphaProduct'" source="main.go:194"
6time="2021-01-29T12:18:19Z" level=info msg="Recorded 1121 results for organization 'AlphaProduct'" source="main.go:203"
7results as metrics" source="main.go:214"

To ensure metrics are indeed being published and scraped, you can access Prometheus at http://localhost:9090 and query it for the snyk_vulnerabilities_total metric.

Start monitoring your vulnerabilities!

All that’s left is to start vulnerability monitoring in Grafana.

Grafana is renowned for the beautiful dashboards that can be built for visualizing metrics. The different panels can be used to slice and dice the data in any way you want.  Of course, building these dashboards is not always easy and so to help you hit the ground running, you can use this ready-made dashboard.

Within Grafana, go to Dashboards | Manage, and click the Import button. You can paste the JSON itself or upload the .json file.

Once imported, open the dashboard to view results!

The value of extensibility

Integrating security into the development workflow in an automated and seamless manner reduces friction and helps gain the trust needed to get developer buy-in. APIs play a key role in this process as they enable the extension, automation and customization of security processes into development workflows.

DevSecOps calls for developers to take more ownership for security. This can’t happen without the right type of tooling and proper guidance from the security team. Security solutions providing extensibility and API provide the flexibility needed to tune automation to fit into existing workflows, resulting in better developer adoption and ultimately - better secure applications.