Best practices for secrets management in serverless applications

If you’re building a serverless application, chances are that your functions need to access secrets or other types of sensitive information that you’re storing, such as API keys, tokens, or passwords. However, managing these secrets properly may sometimes prove to be a difficult task.

When users fail to adopt a key management service, these secrets, unfortunately, end up being stored in source code or in manifest files. The implications of that can be devastating, such as leaking secrets through source code exposure and manipulation, or additional complexity in managing secrets such as rotation and deprecation of keys.

You're welcome to download the PDF for the Serverless security cheat sheet, or read the online blog version.

Following is an example of an insecure way of handling secrets:

service:
 name: hello-world

provider:
  name: aws

functions:
  helloWorld:
    handler: helloworld.get
    environment:
      GITHUB_API_KEY: ABCDEF

  goodbyeWorld:
    handler: goodbye.get

First, let’s review the positive aspects of this structure. The helloWorld function makes good use of the principle of isolating functions. The GITHUB_API_KEY environment variable is only available to the function being invoked, and not to any other function in the code, in this case—goodbyeWorld.

On the other hand, the secret token for GITHUB_API_KEY is hard-coded in the serverless.yml manifest file which mixes issues surrounding configuration and secrets together, leading to potential information exposure if this manifest file is published publicly or shared with other users.

The following example uses the AWS Simple Systems Manager, designed to fetch secrets that were securely encrypted by the AWS Parameter Store. In our example, the SSM variable is used to access a specific value that was created beforehand, making it available for a function through an environment variable:

service:
 name: hello-world

provider:
  name: aws

functions:
  helloWorld:
    handler: helloworld.get
    environment:
      GITHUB_API_KEY: ${ssm:/github/api-key}

Note that ${ssm:/github/api-key} returns the encrypted value of that key and if we wanted to decrypt and return the actual value, we would need to specify ${ssm:/github/api-key~true}.

More information about the Parameter Store and the Key Management Service (KMS) can be found in Amazon’s parameter store documentation.

For even more improved security and flexibility in managing secrets for your applications, consider accessing the secrets and other configuration information in runtime instead of using environment variables that require a process restart to re-apply the new configuration.

Rotate keys and credentials regularly

While using storage for your secrets reduces the chance of a key leaking, it doesn't completely eliminate it. Fortunately, if you're using secrets storage in your function, this means nobody cares what the key actually is... and so you can rotate the key regularly! This way, if the key leaks or is stolen, it can only be used for a short period of time before it simply expires.

Rotating keys is an easy task and can be executed routinely when you are leveraging a KMS. However, it can sometimes prove a bit trickier when integrating with other systems. When the key is necessary for accessing a third-party system, explore the API available by that third-party for a way to rotate the key automatically. If one exists, create a routine job that creates a new key, updates the token in the KMS, and expires or blacklists the old key in the next iteration. In this way, at any given time, only two keys are active. Because your functions are extremely short-lived, they pick the new key rather quickly, allowing you to expire the old key and start another key rotation often—daily or even hourly.

Summary

In closing, it is advised that you use a secure storage solution for your secrets and sensitive credentials: one that is supported by big cloud and FaaS vendors. As an alternative, users can deploy their own solutions, such as Hashicorp’s Vault. Storing keys in a secret storage mitigates the risks of sensitive information stored in static files within a source code repository or environment variables, and greatly reduces the chance of sensitive information exposure. To learn more about secret management tools and best practice click here.

Open a free Snyk account now to improve security in your projects.