Fix vulnerabilities with confidence, with Snyk’s ML-powered Merge Advice

We’re excited to announce the beta release of Merge Advice — the latest enhancement to Snyk’s remediation capabilities that help you fix vulnerabilities reliably. Powered by machine learning, Merge Advice gives developers greater confidence to fix security vulnerabilities in their open source dependencies.

Investing in developer confidence

The risk of introducing breaking changes to a branch means clicking that merge button can feel a bit like playing Russian roulette. Doing something as simple as upgrading a package can sometimes break code that’s reliant on that dependency.

Merging pull requests with confidence is vital, so we’ve continued to invest in making Snyk’s fix and upgrade pull requests accurate, making as few changes as possible to get rid of the vulnerability. For security fixes, for example, we will always recommend the lowest non-vulnerable version compatible with your project. Merge Advice takes this a step further, leveraging machine learning so that more vulnerabilities are able to be confidently fixed.

Each compatible pull request initiated by or through Snyk will show a badge indicating whether or not the fix is likely to result in any breaking changes. Developers now have the intelligence they need to make data-informed merge decisions.

How does it work?

A lot of data crunching happens to determine the merge advice for each PR:

1. PR activity is tracked in both public GitHub repositories and those monitored by Snyk.

2. Three metrics are measured: CI test results, merge rates, and rollbacks for PRs.

3. Machine learning then predicts the success of a PR merge.

The result of this process is a Merge Advice badge displayed at the top of compatible pull requests:

Providing accurate predictions with the help of machine learning

The algorithm powering Merge Advice is designed to predict the success of merge pull requests with high confidence. It relies on a statistical model that takes into account both the number of data points and their observed distribution for an upgrade of a specific dependency from version X to version Y. Advice will NOT be displayed for an upgrade if there is not enough data to make a trustworthy prediction.

The model used by Snyk’s Merge Advice takes a unique approach, as it tracks multiple metrics:

• the pass/fail ratio of CI test suites upgrading a package from version X to Y,

• the merge ratio of those upgrades,

• and the number of times an upgrade was rolled back.

These metrics are also tracked across multiple data sources. We monitor activity in public GitHub repositories with the help of GHArchive — a project that records and archives public GitHub activity and makes it accessible for further analysis. Of course, we also rely on the (currently) over 750,000 repositories secured by Snyk users.

The combination of these data sources creates a rich dataset from which to infer, adding statistical depth to the model used by Merge Advice.

To merge or not to merge — that is the question!

So what does this look like in your day-to-day GitHub workflow? Let’s take a closer look.

As always, Snyk users can manually open a PR to fix a specific vulnerability, or set of vulnerabilities, via the Snyk UI. Snyk can also automatically trigger a pull request in the following two scenarios — when identifying a new vulnerability for existing dependencies or when identifying an out-of-date dependency.

Whatever the trigger, these pull requests are now populated with the new Merge Advice badge. The advice displayed on this badge changes based on the predictive analytics model we described above.

Go fix!

The goal of Merge Advice is to empower developers to make data-informed decisions when merging pull requests for securing their open source dependencies. To ensure the best results, we are constantly improving the algorithm powering this feature. Variables like seasonality, for example, can impact results and will be factored into the calculations as well.

At the moment, Merge Advice badges are visible only for a small portion of GitHub pull requests on Yarn and npm packages where a single package is being upgraded. As we progress with the beta, we will introduce wider and deeper support, so stay tuned for updates.

Merge Advice is available for all users across all plans, including the Free plan.

Happy merging!