Manage multiple Terraform environments

Editor's note: This post originally appeared on CloudSkiff.com. CloudSkiff joined Snyk in October 2021.

How do you manage the complexity of having multiple Terraform environments? With several environments and potentially multiple teams, things can get tough. Here, I’ll walk you through how to get started with managing multiple Terraform environments like a pro.

Getting started with TF files

If you need to manage several Terraform environments, there are a lot of ways to ramp up your approach and get started.

Getting started step by step with single .tf files — the task can be daunting to know all the good practices at first, and probably you might get at best distracted, and at worse discouraged. So you can get started very naively with one single TFState. Create a simple Terraform file, call it production.tf, write your VPCs, your VMs or whatever in it. Very soon you will be able to create another environment. Let’s call it staging.tf. It is going to be still a single TF state file which is not bad, but it won’t scale. But that’s one way to start doing it step by step.

Use Terraform workspaces

Hashicorp recommends you to use what they call now workspaces. workspaces is a Terraform subcommand. It was previously named environments.

The goal of workspaces is to separate one TF state file per environment. So let’s say you have a QA or a staging and a production environment: using that Terraform workspace subcommand, will switch TFStates. That is the recommended way to do things by Hashicorp, but it does not mean that you have to do it exactly that way.

Organize with folders in your Git repo

Another way to do it is to split your state files not using workspaces but by using folders. Just create folders in your Git repo, give each folder a name, like staging and production and just generate different TFState files from those folders. It’s really easy, and then then you are already using splits environments “TFState-wise”.

Modules: A common way of managing Terraform environments

Using modules is kind of becoming a standard now. It is a bit advanced, but modules basically work by injecting variables in them like strings, etc… they are a very handy feature for Terraform.

Modules contain generic code. Let’s take a standard VPC as an example. This VPC can take several values like the subnetwork and a name. What you can do is create two folders, one for each environment. Your first Terraform file will specify what address space your staging VPC will have, while the other will have another address space. So you solve that issue just by injecting different values in modules.

That would be the main way I would suggest using environments.

Get started easily, natively with a single TFState, create different folders, split the states by folders, use modules to inject different types of values in them, or basically just stick to what Hashicorp suggests, like using workspaces.

Organize your resources and define your directory layout

Last thing to consider is how you will organize your resources and directory layout. Would you use one or several repos?

There is no one answer to that, but one common setup is to have your modules handled separately and called from a single repository. Let’s take an example: if you have a module that knows how to properly configure a VPC from two variables, then you have some proper Terraform code. This code can be stored in a distinct Terraform Git repo, managed, versioned, and released like any other project. But still, it cannot be used “as is”. You still need your infra repo to call this module, a bit like you would do with a standard library in engineering.

For some people, the folder hierarchy matters too. It’s very common to see test folders for all the testing, separate folders for the modules, and other folders for the environments as well.

One last common pattern that I think of, is the use of variables files with all the different kinds of values you can have, so that environments can be sent to the execution using those environment variables.

Securing your Terraform code

Snyk IaC secures your Terraform configs (and Kubernetes, CloudFormation, and ARM templates!) as you code, with guided fixes so you can merge and move on. You can test as you write, monitor for changes in your git repositories, and automate testing in your build pipelines before deployment. Getting started with a Free plan takes minutes, while a breach from an IaC misconfiguration can cause damage to last a lifetime. Sign up and start securing your configs below.