Automating Terraform security in Scalr deployments with Regula [Tutorial]

Introduction to Regula and Scalr Integration

Regula

Regula enables cloud teams to evaluate Terraform, CloudFormation, Azure Resource Manager, and Kubernetes Infrastructure-as-Code (IaC) for security and compliance violations prior to deployment. Regula is an open source implementation of Rego, the query language used by the Open Policy Agent (OPA) project. Where relevant, Regula’s policies have been mapped to the Center for Internet Security (CIS) Amazon Web Services (AWS), Azure, Google Cloud, and Kubernetes Foundation Benchmarks to allow users to enforce these policies on IaC prior to deployment.

Regula’s ability to scan IaC for misconfigurations prior to deployment extends the responsibility of cloud security and compliance to development teams in a developer-friendly way, thus removing the security bottleneck when deploying cloud infrastructure using IaC. Regula is maintained by Fugue engineers.

Scalr

Scalr is a Terraform Automation and Collaboration (TACO) software that has support for pull request automation, native Terraform CLI, or module-driven workflows. Scalr helps organizations of all sizes scale through its hierarchical model that centralizes admin operations like RBAC, policy control through OPA, operational views, and a private module registry, resulting in controlled decentralization of Terraform operations allowing developers to execute Terraform workflows within environments independently.

Goal

Pair Regula's convenient, powerful IaC scanning capabilities and Scalr's Terraform automation and collaboration capabilities to demonstrate how organizations can automate the secure deployment of cloud infrastructure with Terraform.

Prerequisites

If you want to follow along, you’ll need the following:

• A Scalr account with custom hooks enabled (note: this is a paid feature). See below for how to configure these.

• A workspace configured in your Scalr account. A workspace is where all objects related to Terraform-managed resources are stored and managed.

• Cloud provider credentials added to your Scalr account (for this demonstration, I will be using AWS)

• A version control system (VCS) provider configured in your Scalr account (for this demonstration, I will be using GitHub)

• Assign scripts/security.bash as the “Before plan” custom hook and scripts/validate.bash as the “After plan” custom hook

• The latest version of Regula in your /usr/local/bin directory (our security.bash script will pull this and run it within the Scalr pipeline, but having it locally will allow you to evaluate your code prior to committing)

Optionally, you can also clone my repository to save yourself some work by executing the following command in your Terminal:

git clone https://github.com/fugue/fugue-scalr-integration.git

What’s in the repo?

What you see above is a visual representation of the file structure for my repository, which most notably contains:

• main.tf: A Terraform file that declares provider and module information

• s3/: A subdirectory that holds Terraform files that contain intentional vulnerabilities

• waivers.rego: A file that waives and disables select rules

• .regula.yaml: A Regula configuration file that declares how I want Regula to be run in this repository

• scripts/: A subdirectory that contains the custom hook described above

Here is a better view of the contents of the S3 module, along with an overlay of the configuration status of all infrastructure (violations of CIS Benchmarks are in red):

Configuring Scalr custom hooks

Custom hooks are used to customize the core Terraform workflow, whether with commands, scripts, API calls. In addition to using Regula for security and compliance scanning, the shell scripts used in this demonstration include checks on Terraform formatting and validation.

Custom hooks can be added upon creation of a workspace or anytime thereafter by going into the workspace settings and specifying what should be run before and after plan; and before and after apply. See below for a demonstration of how to set up custom hooks:

Within the “Advanced” options in the “Settings” page, Scalr enables users to choose the relative path in which Terraform commands will execute (I selected the s3/ directory for this example), which is why the path to the scripts is preceded by “./../”.

Automating security scans along the path of least resistance

The path of least resistance to the cloud through IaC leads beginners and experts alike to Terraform code found on the internet or distributed among colleagues and teams with the assumption that the code is secure and compliant. Such code may pass every formatting, linting, and validation check available, but would expose those who use it to potentially devastating vulnerabilities.

Exposing latent vulnerabilities: scanning infrastructure with Regula

For this demonstration, I purposefully took this path of least resistance by seeking Terraform code that would allow me to deploy an S3 bucket as quickly as possible. I’ll begin by showing you how vulnerable this Terraform code is by executing regula run in this repository locally:

When you execute this command, Regula will:

• automatically detect supported IaC files throughout the entire repository

• run a security and compliance scan against all detected, supported IaC files

• Output the results of that scan in descending order of severity, with the following information:

  • Fugue rule ID (e.g. FG_R00099) and title

  • Rule severity (e.g. [High])

  • Hyperlink to rule remediation steps (e.g.https://docs.fugue.co/FG_R00099.html)

  • Location of rule violation listed by IaC file

The results of the scan above should worry DevOps and Security Engineers alike. By copying, pasting, and deploying vulnerable code, we are exposing ourselves to hackers who use automation to detect and exploit these vulnerabilities. Furthermore, I voluntarily conducted this scan without integrating it into a pipeline (such as Scalr’s pipeline) – not quite the path of least resistance! Past breaches teach us that cloud hackers find vulnerabilities using automation, then use those vulnerabilities (such as a vulnerable S3 bucket) as footholds to strike at the infrastructure and data that needs to be protected the most. Failure to use similar automation to properly configure resources before they get to the cloud makes such hacks a matter of “when,” not “if.”

Let’s dig deeper into these rule violations by taking a look at a snippet of the code (if you’re following along, the code below is from lines 18 - 26 of s3/bucket.tf, and the indentation is adjusted left for easier viewing):

18server_side_encryption_configuration {
19  rule {
20    apply_server_side_encryption_by_default {
21      #Un-comment below to satisfy FG_R00099
22      #kms_master_key_id = aws_kms_key.mykey.arn
23      #sse_algorithm     = "aws:kms"
24    }
25  }
26}

I have paired the intentional vulnerabilities in this code with comments as hints for where to revise the code to adhere to the numbered rules that come standard with Regula. As it is currently written, the code above violates rule FG_R00099 (a high severity violation according to CIS Benchmarks): if deployed without being fixed accordingly, this AWS Simple Storage Service (S3) bucket would expose data at rest. For the beginners reading this post, this would be akin to leaving a bank vault and safety deposit box open. Even if you were to make your deposit with an armored truck (transit encryption, in this analogy), failure to lock the vault and safety deposit box (apply server side encryption) could make such attempts at data security futile.

The scary part of this example is that the Terraform documentation from which I derived this code describes how to deploy an S3 bucket as simply as possible (as documentation tends to do), prioritizing examples of required parameters, leaving optional parameters (server side encryption being among them!) for the easily overlooked latter portions of documentation. The configuration of your cloud resources, after all, is your responsibility: your cloud provider merely provides you the means with which to securely deploy and use cloud infrastructure.

Why leave security up to chance when you can so easily automate it?

Applied flexibility: waiving and disabling rules

Critical business needs sometimes conflict with low and medium severity rules. For example, a business that depends on delivering content via an S3 bucket that hosts a static website would understandably grow tired of seeing that the S3 bucket in question is in violation of FG_R00229 (S3 buckets should have all “block public access” options enabled). Regula users are able to waive specific rules for specific resources or disable certain rules altogether. For this demonstration, I have decided to waive FG_R00274 for my logging bucket and disable Fugue rule FG_R00275 altogether through the use of my waivers document (waivers.rego). See below for how simple waiving and disabling rules is with Regula:

package fugue.regula.config

waivers[waiver] {
  waiver := {
    #Waiving bucket logging (for the logging bucket)
    "rule_id": "FG_R00274",
    "resource_id": "module.s3.aws_s3_bucket.logbucket"
  }
}

rules[rule] {
  rule := {
    #Disabling cross region replication (budgetary purposes)
    "rule_id": "FG_R00275",
    "status": "DISABLED"
  }
}

Operationalizing IaC security: stopping misconfigurations from getting to the cloud

Now let’s get into how Regula pairs with Scalr’s Terraform remote state and operations backend to stop misconfigured infrastructure from being deployed to the cloud.

The first custom hook script (security.bash) pulls, extracts, moves, and executes the latest version of Regula, scans all Terraform files (HashiCorp language (HCL) or Terraform JavaScript Object Notation (JSON) plans), and produces either a non-zero exit code and message that stops the Scalr build from continuing; or an exit code of zero, allowing the Scalr build to continue.

The next custom hook script (validate.bash) ensures that the Terraform in this repository is valid and properly formatted according to HCL canonical standards. If the Terraform is not valid, this script produces a non-zero exit code. If the Terraform is valid, Scalr executes the rest of the build, automatically running terraform plan and terraform apply, thus deploying your infrastructure to the cloud and maintaining the Terraform state in its easy-to-use interface.

Trying (and failing) a build

Next, I will show you what it looks like when I commit vulnerable Terraform code to my GitHub repository. I begin by executing the following commands (<files> is a placeholder for which files should be committed and pushed to GitHub):

git add <files>
git commit -m "initiating the scalr terraform pipeline"
git push

Scalr will detect that I have commited changes to my repository, and will use the scripts I have provided to scan my Terraform for security and compliance issues:

Resolving configuration issues with Regula

Now that I know I have misconfigurations in my Terraform files, I can go back into my repository in VSCode and execute a regula run locally to address those issues (alternatively, I could export the output of the regula run that occurred in my Scalr run). I set up this repository to allow me to un-comment my Terraform code corrections easily, but properly configuring your infrastructure is as easy as clicking the hyperlink that populates with every rule following a regula run. Notice that both rules that I am fixing (FG_R00036 and FG_R00101) are single lines of code; these lines of code are pulled directly from the rule remediation steps contained within the hyperlink provided in the output of this command. You can’t get much more developer-friendly than that.

Trying (and succeeding!) a build

With my infrastructure properly configured, I'll commit to my GitHub repository again to maximize Scalr's Terraform automation capability. This will again run my custom hook scripts, then (if those are successful) Scalr will run terraform plan and terraform apply.

To demonstrate this, I'll re-run the commands I ran initially:

git add <files>
git commit -m "initiating the scalr terraform pipeline"
git push

...resulting in a successful build!

And that's it! Now, thanks to Regula and Scalr, we have a pipeline to securely automate the deployment of cloud infrastructure using Terraform.

Next steps

The example above shows how to secure your IaC according to CIS Benchmarks, but for those who need other compliance families (such as HIPAA, SOC2, ISO 27001, NIST 800-53, GDPR, PCI DSS, AWS WAF, or CSA), please visit www.fugue.co. The example above also shows custom hooks (a paid Scalr feature), but for those who desire more quality of life features such as cloud infrastructure cost estimation, self-hosted agents, SSO, or policy previews, please visit https://www.scalr.com/.

Want to learn more about Regula? See our GitHub repo and documentation. Regula evaluates Terraform HCL, Terraform plan JSON, CloudFormation YAML/JSON, Kubernetes YAML manifests, and Azure Resource Manager templates for security and compliance. Regula also supports waivers, custom rules, enabling or disabling rules, and more.

Interested in other ways to automate the secure deployment of cloud infrastructure? Check out our blog post about integrating Regula and Travis CI, or our blog post about integrating Regula and Bitbucket Pipelines.