How to choose a Software Composition Analysis (SCA) tool

Whether you’re a developer or a security engineer, Software Composition Analysis — or SCA for short — is a term you will start to hear of more and more. If you haven’t already, that is.

The reason for this is simple.

Your company is increasingly relying on open source software and containers to develop its applications and by doing so is introducing risk in the form of security vulnerabilities and license violations. The application security methodology known as Software Composition Analysis (SCA), together with specialized tools that enable its practice, can help your organization manage and mitigate this risk.

But even if you and your organization understand the risk and want to put proper controls in place by using an SCA tool, how do you choose the right one? There are plenty of SCAtools out there—are they all alike or are there important differences between the various offerings? What are the key capabilities that are absolutely essential for successful mitigation of the risk introduced by open source and which ones can be looked over? What role should a vulnerability database play in the decision making process? How do containers factor in?

To help guide you in making your decision, we put together a cheat sheet that summarizes the answers to these questions. For more context into what SCA is and why it’s important, as well as a deeper understanding of the key requirements to look for in an SCA tool, read on!

What is a Software Composition Analysis (SCA) tool and why do you need one?

Adoption rates for open source are sky high. The open source trend, already gaining momentum well before COVID19 hit, has now accelerated. Gartner estimates that 90% of organizations rely on open source in their applications today which might even be an underestimate.

Open source enables companies to develop faster and thus remain competitive. It does this by enabling developers to save the time—and organizations the money—it would have taken them to build functionality from scratch. Open source has other benefits of course, such as being flexible and free, and helping companies avoid vendor lock in.

But just because something is free, that does not mean there is no cost involved. In fact, from a security and compliance perspective, open source can turn out to be very expensive. That is if the risk introduced by the open source packages being used is not managed properly. Open source components contain known security vulnerabilities that need to be checked. They also contain open source licenses with terms and conditions that may restrict usage. Without visibility into what open source is being used and without understanding the scope of the risk this poses, can result in disastrous, and costly, consequences.

Applications today include much more than open source. To help developers build, test, and deploy applications more quickly, they are being packaged into containers which in turn are orchestrated by container orchestration tools such as Kubernetes. Suddenly, the list of components that compose an application and that need to be secured is longer and more complicated. 

In comes SCA, an application security testing methodology that helps development and security teams to track the different components being used and detect the security vulnerabilities they contain.

What is a Software Composition Analysis (SCA) tool?

An SCA tool facilititates the effective management of the various components comprising applications. Using an SCA tool, development teams can quickly track and analyze any component brought into a project, including open source components, their supporting libraries, their direct and indirect dependencies, and containers. SCA tools can also detect software licenses, deprecated dependencies, vulnerabilities and potential exploits. The scanning process generates a bill of materials (BoM), providing a complete inventory of a project’s software assets.

10 key requirements when selecting an SCA tool

SCA is most definitely not new—it has been around for at least a decade. But the rising popularity of open source, as well as the changes to the way applications are assembled from multiple components, have contributed to the forming of a packed market consisting of many different type of solutions. These range from open source scanners, to specialized commercial tools, to AppSec suite solutions. Some software development and delivery solutions also provide basic SCA capabilities.

Overwhelmed? Don’t be. To enable you to successfully manage AND mitigate the risk posed by open source, evaluate an SCA tool using these key requirements.

1. Developer-friendliness

2. Ecosystem support & integrations

3. Dependency analysis

4. Vulnerability detection

5. Prioritization

6. Remediation

7. Governance & control

8. Reporting

9. Automation & extensibility

10. Cloud native application security

1. Developer-friendliness

The time and age where security simply hands over a list of vulnerabilities for developers to fix is long gone. That type of siloed approach is no longer tenable. DevSecOps calls for developers to take more ownership for security but that cannot be achieved if the tools they use work against them instead of for them.

Developer adoption is key. If an SCA tool is too difficult to use or hampers development, it will not be used by developers and will not be of much use. An SCA tool, therefore, needs to:

• be intuitive and easy to set up and use. Developers will appreciate tools that work the same way their existing tools work.

• easily integrate with existing development workflows. A tool that is just click or two from integrating into a git-based workflow, or that can be easily plugged into an IDE or CLI, will go a long way in driving developer adoption.

• provide automated and actionable advice. An SCA tool that not only surfaces issues but actually also guides developers on the path to remediation with automated and actionable fix advice empowers developers to take action. Snyk’s automated fix pull requests is a good example of this.

2. Ecosystem support & integrations

Without the ability to cover the languages being used to build your applications or fit into your development environment, an SCA tool is not going to be very helpful, right? This sounds like a pretty basic requirement but should not be taken for granted. Some SCA solutions might provide full language coverage but will not provide a Jenkins plugin to enable you to easily add application security testing as a step in your build process. Others might not provide IDE plugins which enable you to shift security far left in the SDLC.

For language support, verify that an SCA tool is able to provide security coverage for the core programming languages and frameworks used to build your applications.

Tip!Some of the languages in your technology stack might not need to be prioritized initially. Try evaluating an SCA tool based on the support it provides for the core languages you are using.

For integrations, it is not only about breadth and being able to integrate across the SDLC, but also depth. Be sure that the SCA tool provides both an easy integration and one that actually provides results as expected. We will discuss API later, but the availability of a robust API is a big advantage here.

3. Dependency analysis

As reported inSnyk 2020 State of Open Source Security report, 80% of vulnerabilities in open source packages are identified in transitive dependencies!

This means that the vast majority of vulnerabilities in your code base are introduced by dependencies you had no idea you were using in the first place. These transitive dependencies can go several layers deep, making it extremely challenging to gain end-to-end visibility into what open source is actually being used by an application.

Not only that, to accurately identify the dependencies an application is using, as well as the vulnerabilities they introduce, a deep understanding of how each ecosystem handles dependencies is required. Package resolution during installation, lock files, development dependencies—all these are examples of factors that affect how vulnerabilities in open source packages are identified and will determine subsequent remediation steps.

And so when evaluating an SCA tool, it is important to verify that it can accurately interpret all the dependencies in an application to provide practitioners with full visibility.

4. Vulnerability detection

An SCA tool must be able to accurately detect whether an open source package contains vulnerabilities or not. This depends, as discussed above, on the ability of the tool to understand the dependency logic, but just as importantly, on the security data the tool relies on.

It is here that the differences between the SCA tools come more to light. Some SCA tools will rely solely on public databases, such as NVD. Others might augment public databases with additional publicly available vulnerability information. Some SCA tools—Snyk included—maintain a security team that combines multiple sources into one vulnerability database, continuously updated, and enriched using various advanced analysis processes. Even there, there are nuances from solution to solution, pertaining to the quality of the database and the accuracy, and comprehensiveness of the intelligence it provides.

General speaking, when evaluating the vulnerability data an SCA tool provides, we recommend considering the following:

• Accuracy - False-positives are unavoidable but a high rate of false-positives will waste resources and hamper developer adoption.

• Comprehensiveness - Relying on NVD alone is not enough. Be sure to select an SCA that combines public, private and internal security data sources. 

• Timeliness - The speed at which newly disclosed  vulnerabilities are added into a database if critical. Can you afford not to identify when a package you are using contains a zero-day vulnerability?

• Actionability - Select an SCA tool that provides rich and contextual information on vulnerabilities to help development take action. 

5. Prioritization

The number of vulnerabilities in open source components is constantly on the rise, with thousands of new vulnerabilities disclosed every year. SCA tools will often identify hundreds if not thousands of vulnerabilities that quickly pile up into backlogs that can easily overwhelm teams.

Since you cannot, realistically, fix all the vulnerabilities on the list, you need to decide which vulnerabilities offer the best return for time invested. These decisions will have a major impact on your effort to manage and reduce risk. Bad prioritization, leading to time wasted on false-positives, can cause friction and reduce developer trust which we already said was critical for DevSecOps and scaling security.

At the very least, make sure that an SCA tool can help you prioritize well by:

• going beyond CVSS scoring for risk assessment. CVSS is a good starting point but that’s about it. It lacks context and is often difficult to understand and use.

• providing deep application-level and business-level context on vulnerabilities. Context is king. Can the SCA inform you whether a vulnerability is reachable or not? Can it tell you whether it is exploitable? Does the SCA help you understand whether the vulnerability is in a component being used in production and by a mission-critical application?

• automating prioritization. At scale, manually prioritizing is simply not possible. SCA tools that help you automate processes across projects and teams with policies should be considered over those who don’t.

6. Remediation

The vast majority of SCA tools provide the ability to identify security vulnerabilities in open source packages. While some go beyond this to support taking the next logical step—the remediation of vulnerabilities—remediation capabilities vary from tool to tool. 

Consider remediation advice as an example. It is one thing to suggest an upgrade for a dependency to a version fixing the vulnerability in question. It is another to calculate the minimal upgrade path so as not to risk breakage. And it is entirely a different thing to automatically trigger a pull request when a new vulnerability is identified with the recommended fix.

When evaluating, dive deeper into the remediation advice the SCA tool provides around a vulnerability and the workflows it supports to drive actionability. Is there enough information available for understanding where and how to apply a fix? Are automated workflows available?

7.  Governance & control

Does the SCA tool provide you with the control you need to control the use of open source in your applications? At the very least, ensure the SCA tool provides granular policies for defining and automatically enforcing the security and compliance guidelines accepted by your organization.

8. Reporting

Being able to keep track over time of the various open source packages being used across the organization, including the various open source licences they contain, is important for various reasons and different business stakeholders. For example, security leaders will want to measure the success of SCA processes over time by answering how many vulnerabilities were identified and how many were remediated.

Compliance and legal offices will most likely be interested in generating a BoM report, for an inventory on all the open source dependencies and licenses that impact the compliance posture of the organization. Verify an SCA tool provides you with the oversight needed to track your posture over time and enable you to generate, and share, a BoM report on your open source inventory. 

9.  Automation & extensibility

The larger you grow, the more challenging it is to perform all the manual operations involved in SCA processes. The ability to automate tasks such as adding new projects and users to be tested, or scanning new builds as part of your CI/CD pipelines, drives efficiency but also helps reduce friction with existing development workflows—a key ingredient for successful DevSecOps.

One key requirement to consider is the existence of a robust API that enables the automation, customization and integration of SCA processes into your existing workflows and systems. Do you already have systems set up for security monitoring? An API should enable you to integrate results into these systems. Are you using build tools as part of your continuous delivery process? Make sure the SCA tool provides plugins for automating security testing as part of the process.

10. Cloud native application security

Modern applications are composed of multiple components that all need to be scanned and secured to provide end to end security coverage. In addition to open source dependencies—containers, infrastructure as code, and proprietary code are the building blocks used to assemble applications today.

For container security, an SCA tool should be able at the very least to scan container images for security vulnerabilities and integrate into the workflows, tools and systems used to build, test and run them. More advanced solutions provide remediation steps as well. Snyk, for example,  identifies the container base image and recommends upgrades that will reduce vulnerabilities and also provides the container build commands and dependencies that introduce vulnerabilities, to simplify and speed up container remediation.

The bare minimum

Every organization has different needs that will vary based on various factors such as their technology stack, use case and of course—their available security budget and priorities. There is no such thing as a fit-all SCA tool.

But at the very least, and if pressed to make a rushed decision, be sure that the SCA tool you choose is able to accurately detect all the open source components you are using AND any vulnerabilities and licenses they contain.

Of course, to ensure the tool is actually being used by developers—the main end user—just as important is developer-friendliness. The best vulnerability database in the world, for example, goes to waste, if the SCA tool it powers is not being adopted. Select an SCA tool developers understand and know how to use, and might I dare say,even enjoy using!

Deliver fast, stay secure!