Testing effectively in Terraform

Terraform is an infrastructure as code (IaC) solution that enables DevOps teams to use coding concepts to automatically deploy on-premise or public cloud infrastructure components. These components may include virtual machines, network components, storage, applications, or database services.

Terraform provides HashiCorp Language (HCL). HCL is a unified, domain-specific language that streamlines the authoring of templates across different cloud providers, including Amazon AWS Cloudformation, Microsoft Azure ARM Templates and Bicep, and Google’s Cloud Deployment Manager for GCP.

Just as testing code is a crucial component of software development, testing an IaC solution like Terraform is vital. When properly integrated, IaC testing can mitigate significant resource management issues and enables DevOps teams to validate the effectiveness of configurations.

However, integrating this testing into the DevOps process can be challenging. Questions arise as to what this testing should include, what it can omit, and which available mechanisms are most effective. 

Testing Terraform templates can be done many ways. Using Terraform's built-in commands, like terraform validate or terraform plan, is easiest. These commands enable us to write unit, integration, compliance, or end-to-end tests — which aligns well with traditional software development testing practices. Alternatively, we can take a more developer-oriented approach by testing with the Cloud Development Kit for Terraform (CDKTF).

Ultimately, DevOps teams need to be clear about the goals of the validation process. These may include verifying the template syntax and ensuring the success of template deployment. Or the goal may be to develop an altogether more advanced testing scenario: doing focused testing for security issues or misconfigurations, and detecting security implications when running deployments. The principal idea is to determine exactly what must be tested, and then find the proper combination of testing solutions.

Terraform testing strategies

Let’s explore some different testing strategies for Terraform.

Drift testing

Terraform validates your current infrastructure using the terraform state file. Whenever a Terraform deployment occurs, the end state gets written to a state file, which is typically stored in a shared location.

However, if not all deployments in your target environment are triggered from Terraform, you might end up with a difference between the “real” resources and what the Terraform state file thinks you have running. When this deployment affects a resource already managed by Terraform, the built-in Terraform command terraform plan is useful, as it’s capable of highlighting any changes or differences between what’s in scope of its state file and the actual target environment. When this deployment creates an entirely new resource, tools like driftctl and CloudQuery are useful, as they report all the resources that are not in Terraform control.

Unit testing

Unit testing is one of the easiest types of testing to perform. It entails running a specific and relatively basic test — typically against an individual object. Often, this includes verifying that the template syntax is correct.

For this, we can use terraform validate. If there’s a mistake in the template syntax, terraform validate returns an error message (although having a successfully flagged syntax template doesn’t necessarily guarantee successful deployments).

To validate a deployment, we can rely on another built-in command: terraform plan. By running the plan phase, Terraform simulates an actual deployment, communicating with the provider's back end and reporting the outcome of the deployment using the terraform apply command.

Running the terraform plan output will highlight any changes detected against the existing environment. This might include the creation, modification, or deletion of new resources. This information could help our DevOps team identify the impact of running the deployment before they execute it.

The next step is using Terraform apply, which runs the actual deployment. Keep in mind, though, that this phase doesn’t always fully reflect the situation. If you remove a resource from your Terraform HCL and then use terraform apply, it will delete the resource on the IaaS — because while that resource is no longer in the HCL, it is still found in the Terraform state from when it was last deployed correctly. Terraform compares the TF state to the IaaS API, finds that the resource should not be deployed anymore, and deletes it. So you still need to watch for any possible dependencies or constraints in your environment. In cloud scenarios such as AWS or Azure, this is typically managed by policy governance. 

Linting

Using a linter is also helpful. A linter enables DevOps teams to analyze code for errors, bugs, or suspicious code blocks. This helps identify syntax issues, but it also has more critical use cases, like detecting security issues.

There are several open source Terraform-linter tools with different analysis mechanisms for various target platforms — GitHub provides an assortment of templates, using tflint against different cloud providers.

Depending on the complexity of the linting use case, the process might qualify as a simple unit test or fit into a more complex or advanced testing scenario. For example, if the template allows a virtual machine deployment with a public IP address attached to it, our linter validation might detect the presence of the public IP address object and throw an error. The linter could also detect hard-coded password strings or connection strings in our code.

Integration testing

Even if each unit test passes, running all Terraform plans sequentially doesn’t guarantee overall success. This is why integration testing is essential. Simply stated, integration testing means running a combination of unit tests. In Terraform, unit tests usually target a single module, whereas integration tests validate multiple modules with extensive dependencies.

Suppose we have two separate Terraform templates: one to create a virtual network or virtual private cloud (VPC) and another to deploy virtual machine infrastructure. We could run each as an individual unit test, but eventually, we’ll want to successfully deploy the virtual machines against the virtual network’s deployed components. If either module deploys to a different region, unit testing would still be successful, but integration testing would fail.

Another important difference: unit testing can be run offline without deploying anything, whereas integration testing requires deploying resources to validate impact and success.

One tool to help with integration testing is Terratest by Gruntwork. It’s based on the Go language framework with built-in package testing, enabling DevOps teams to apply the same principles to IaC scenarios.

Compliance testing

When validating Terraform templates, technical deployments must align with corporate compliance regulations. This necessitates compliance testing — checking regional compliance standards, naming conventions, infrastructure standards like specific VM types, and disk storage security.

In most cases, compliance validation is (or should be) part of the target environment configuration, along with integrating Azure policies or the AWS compliance center. However, these policies are only validated during an actual deployment, which will fail if it doesn’t align with compliance policies.

By integrating compliance testing into the Terraform DevOps process, run testing occurs before deployments, ensuring that deployments are successful. One solution is Terraform-compliance, an external tool that validates an organization's compliance with Terraform templates.

End-to-end testing

Ultimately, DevOps teams will try to combine all testing methods into a single scenario. Ideally, we want them to test all the templates, deploying all modules and validating successes or failures.

This sounds ideal, but end-to-end testing poses challenges. The duration of tests causes a bottleneck — it can take much longer to receive an output than with unit tests, which run quickly and provide immediate results.

Unit testing with CDKTF

So far, our examples assume that we’re using native HashiCorp Language for our template files. However, Terraform also offers a more developer-oriented approach to IaC with the CDKTF - Cloud Development Kit for Terraform.

Using the CDKTF, DevOps teams can deploy infrastructure using any traditional development language — such as Python, C#, Go, and Java. Every Terraform provider and module is compatible with this kit. The main benefit is that our team doesn't have to learn HCL and can apply their existing development language knowledge.

CDKTF involves a different approach to writing templates and running deployments, so unit tests will run differently. Up-to-date documentation on CDKTF testing is available on the Terraform website.

Snyk security tools

Testing your IaC is crucial for the health of your environment. In addition to evaluating different testing methodologies, it’s important to highlight the security aspects of testing. Snyk provides helpful tools for this:

• Snyk Infrastructure as Code helps integrate security into all aspects of writing IaC, performing security code scanning in a way that’s similar to traditional application source code.

• driftctl, the previously mentioned drift testing tool, is developed by Snyk, and available as an open source drift detection engine.

• Snyk IaC comes with an extensive set of predefined industry-benchmarked security rules, best practices from cloud providers, and Snyk’s threat model research. We can also build custom rules with Snyk IaC, using the Open Policy Agent (OPA) framework and the Rego query language.

Learn more about security for Terraform

Template testing is a crucial part of the template creation process. In addition to the functional benefits of testing, it helps us optimize security in Terraform deployments. If you’re interested in learning more about integrating security into your Terraform deployment templates, take a look at Snyk, which provides security validation tools for your DevOps and automation pipelines — including Terraform.