Black box testing basics

Black box testing explained

Creating applications that are both high quality and secure is of the greatest challenges of software development. Security testing helps to address both by identifying potential flaws and security holes in software. Black box testing is a good starting point since it simulates how an attacker would exploit flaws in a system in order to gain access.

In this article, we’ll cover everything you need to know about black box testing, including testing types and techniques.

Black box testing diagram

4 Types of black box testing

Security practitioners rely on a number of black box testing techniques — both automated and manual —  to evaluate a system’s security. Let’s review some of the most popular types of black box testing.

1. Vulnerability scanning

In order to hack an application, the attacker must first understand how it works. Vulnerability scanning offers an easy way for hackers to learn about a system and discover security holes. But vulnerability scanning is also an important part of application security, as it allows you to play the role of a hacker in order to prevent such attacks.

While automated black box testing software like ZAP often show vulnerabilities that are more easily caught, more experienced security practitioners can complement this with manual vulnerability testing to discover vulnerabilities automated testing may have missed. You can also use a number of tools together to check for vulnerabilities, for example, supported tools in Kali Linux or the Chrome DevTools for inspecting web applications.

2. Penetration testing

Penetration testing simulates real-world attack scenarios in which hackers attempt to access and collect data in order to perform malicious actions to compromise the system.

Penetration testing is usually executed manually, based on the experience of the penetration tester. For example, penetration testing can be used to check the API that retrieves the available products a user can buy. In this case, you’d need to test it using different input to reveal any abnormal responses or whether any stack trace errors are displayed.

Penetration testing can also be done automatically using open source tools like Powershell Suite

3. Network topology discovery

Network topology discovery helps to understand the current network layout within your system, including how elements are linked together in the network and how they interact with each other. This, in turn, helps to identify potentially vulnerable elements in the network system in order to mitigate risk.

4. Risk assessment

This unique security testing type requires very little technical knowledge about the system, relating more to the business side of things: requirements that might pose security risks. Risk assessment can be applied in the earliest product development stages: from inception and sprint planning of the software development team. It can thus prevent security issues early on in the SDLC.

Black box testing example

Let’s look at an example of black box testing.

First, you’ll need to clone this repository to your local environment: https://github.com/appsecco/dvna. This will trigger the sample application, which has a number of exploitable holes:

To quickly set up the example, run the following command:

docker run --name dvna -p 9090:9090 -d appsecco/dvna:sqlite

Automated vulnerability scanning with ZAP

We’ll be using ZAP to conduct black box testing, so you’ll need to install ZAP on your machine.

First, we’ll execute automated scans to check the site for vulnerabilities. Once the scan is complete, you’ll have a list of alerts for vulnerabilities as well as a description and proposed fix for each.

Manual SQL injection

While automated scanning is easy to perform and a good starting point for security testing, there are security aspects that require manual testing.  SQL injection is one such example. Let’s run the simple SQL ' or '1'='1 to search in the application:

Here we show it has affected the user interface displayed to all system users, which could allow hackers to collect system user data or even sell customer data to competitor companies.

The black box testing process

Black box testing checks systems for security issues that could be exploited, without the need to access the software product code or to have an in-depth understanding of how the application is being developed. After the test is complete, it provides a list of security bugs to be reviewed, prioritized, and fixed.

The next steps include:

• The development team is tasked with finding the solutions for each security problem.

• The security analysts double check to ensure the bugs have indeed been fixed.

• The development team runs regression tests to ensure everything is fine and that no existing features have been broken due to the fix.

4 Black box testing techniques

Security testing can be viewed as an art form, especially when it comes to black box testing. The fundamental rule here is the need to be creative and think like a hacker. There are multiple black box testing techniques that can be applied.

1. Fuzzing

The fuzzing technique tests API services or web interfaces to check system behavior with random or customized input. If any unusual behavior is detected, the development team must find the root cause and come up with a solution for the fix.

2. Syntax testing

Syntax testing checks for invalid input such as wrong syntax. It also checks if the system is showing any sensitive data related to databases or customer information, which hackers might exploit.

3. Exploratory testing

Exploratory testing is a common black box analysis technique to help security analysts learn more about the system by looking for hidden security issues throughout the security testing journey. Security bugs are found and reported along the way, with suggested fixes.

4. Data analysis

Data analysis testing is related to checking logs, responses from API backend services, or web interfaces that may be illegal or can be used to attack the system or collect data from users. Performing data analysis tests properly and efficiently requires a good monitoring and debugging system to collect logs and visualize data. There may also be a need to set up rules for security alerts for instant notification when security issues arise.

Black box vs. white box testing

When it comes to security testing, there will be cases that require both black box and white box testing be performed. Let’s explore the differences between two of the most popular black box and white box testing methods: dynamic analysis security testing (DAST) and static application security testing (SAST).

Black box method: Dynamic application security testing (DAST)

• Dynamic testing without accessing and analyzing the source code of the application.

• Can be conducted manually (by security experts) or automatically.

• Application is tested from the outside, without accessing the software’s internal structure.

• Difficult to ensure the entire code base has been covered.

• Generally more expensive since the application must be deployed in order to test it.

White box method: Static application security testing (SAST)

• SAST is used to analyze source code.

• Generally executed before the code is compiled.

• Often done using automated SAST tools.

• Capable of analyzing 100% of the code base.

• Less expensive to fix security issues since they are discovered earlier in the software development life cycle.

Whether black box, white box, or both testing types best fit your needs will depend on the use case.

Conclusion

Black box testing is an important part of application security. Without knowledge of the software’s internal architecture, this testing method offers an objective, real-world view of your application. While it may not cover the entire codebase, when combined with other security testing methods, it empowers security teams by helping them to deliver high-quality, more secure products.

Offering developer-first tooling and best-in-class security intelligence, Snyk helps developers deliver quality products faster while keeping your code, open-source libraries, containers, and infrastructure as code secure. Book a demo today.