Protect sensitive input variables

Often you need to configure your infrastructure using sensitive or secret information such as usernames, passwords, API tokens, or Personally Identifiable Information (PII). When you do so, you need to ensure that you do not accidentally expose this data in CLI output, log output, or source control. Terraform provides several features to help avoid accidentally exposing sensitive data.

In this tutorial, you will use Terraform to deploy a web application on AWS, including a VPC, load balancer, EC2 instances, and a database. You will replace the database's hard-coded credentials with variables configured with the sensitive flag. Terraform will then redact these values in the output of Terraform commands or log messages. Next, you will set values for these variables using environment variables and with a .tfvars file. Finally, you will identify the sensitive values in state, and learn about ways to protect your state file.

Prerequisites

You can complete this tutorial using the same workflow with either Terraform Community Edition or HCP Terraform. HCP Terraform is a platform that you can use to manage and execute your Terraform projects. It includes features like remote state and execution, structured plan output, workspace resource summaries, and more.

Select the HCP Terraform tab to complete this tutorial using HCP Terraform.

Create infrastructure

Clone the Learn Terraform Sensitive Variables GitHub repository for this tutorial.

$ git clone https://github.com/hashicorp-education/learn-terraform-sensitive-variables

Change to the repository directory.

$ cd learn-terraform-sensitive-variables

This configuration defines a web application, including a VPC, load balancer, EC2 instances, and a database.

$ terraform init
Initializing the backend...
##...
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.

terraform {
  cloud {
    organization = "organization-name"
    workspaces {
      name = "learn-terraform-sensitive-variables"
    }
  }
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 4.4.0"
    }
  }
  required_version = ">= 1.2"
}

$ terraform init
Initializing HCP Terraform...
Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Installing hashicorp/aws v4.4.0...
- Installed hashicorp/aws v4.4.0 (signed by HashiCorp)
HCP Terraform has been successfully initialized!
You may now begin working with HCP Terraform. Try running "terraform plan" to
see any changes that are required for your infrastructure.
If you ever set or change modules or Terraform Settings, run "terraform init"
again to reinitialize your working directory.

Now apply the configuration to create the example infrastructure.

$ terraform apply

Running apply in HCP Terraform. Output will stream here. Pressing Ctrl-C
will cancel the remote apply if it's still pending. If the apply started it
will stop streaming the logs, but will not stop the apply running remotely.

Preparing the remote apply...

To view this run in a browser, visit:
https://app.terraform.io/app/hashicorp/learn-terraform-sensitive-variables/runs/run-3nZZ9owarvBTaMBs

Waiting for the plan to start...

Terraform v1.2.3
on linux_amd64
Initializing plugins and modules...
module.ec2_instances.data.aws_ami.amazon_linux: Reading...
data.aws_availability_zones.available: Reading...
data.aws_availability_zones.available: Read complete after 0s [id=us-east-1]
module.ec2_instances.data.aws_ami.amazon_linux: Read complete after 2s [id=ami-065efef2c739d613b]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

## ...

Plan: 36 to add, 0 to change, 0 to destroy.


Do you want to perform these actions in workspace "learn-terraform-sensitive-variables"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

## ...

Apply complete! Resources: 36 added, 0 changed, 0 destroyed.

Respond to the confirmation prompt with a yes.

If you use HCP Terraform to provision your resources, your workspace now displays the list of all of the resources it manages.

Refactor database credentials

Open main.tf in your text editor. Near the bottom of the file, find the aws_db_instance.database block that defines your database. The database username and password are hard-coded. Refactor this configuration to remove these values.

First, declare input variables for the database administrator username and password in variables.tf.

variable "db_username" {
  description = "Database administrator username"
  type        = string
  sensitive   = true
}

variable "db_password" {
  description = "Database administrator password"
  type        = string
  sensitive   = true
}

Notice that you've declared the variables as sensitive. Now update main.tf to reference these variables.

resource "aws_db_instance" "database" {
  allocated_storage = 5
  engine            = "mysql"
  instance_class    = "db.t3.micro"
-  username          = "admin"
-  password          = "notasecurepassword"
+  username          = var.db_username
+  password          = var.db_password

  db_subnet_group_name = aws_db_subnet_group.private.name

  skip_final_snapshot = true
}

If you were to run terraform apply now, Terraform would prompt you for values for these new variables since you haven't assigned defaults to them. However, entering values manually is time consuming and error prone. Next, you will use two different methods to set the sensitive variable values, and learn about security considerations of each method.

Set values with a .tfvars file

Terraform supports setting variable values with variable definition (.tfvars) files. You can use multiple variable definition files, and many practitioners use a separate file to set sensitive or secret values.

Create a new file called secret.tfvars to assign values to the new variables.

db_username = "admin"
db_password = "insecurepassword"

Apply these changes using the -var-file parameter. Respond to the confirmation prompt with yes.

$ terraform apply -var-file="secret.tfvars"
random_string.lb_id: Refreshing state... [id=2Mw]
module.vpc.aws_vpc.this[0]: Refreshing state... [id=vpc-05f973211a47fb6f4]

  ## ...

  # aws_db_instance.database will be updated in-place
  ~ resource "aws_db_instance" "database" {
      + domain                                = ""
      + domain_iam_role_name                  = ""
        id                                    = "terraform-20210113192204255400000004"
      + kms_key_id                            = ""
      + monitoring_role_arn                   = ""
      + name                                  = ""
      # Warning: this attribute value will be marked as sensitive and will
      # not display in UI output after applying this change
      ~ password                              = (sensitive value)
      + performance_insights_kms_key_id       = ""
      + replicate_source_db                   = ""
        tags                                  = {}
      + timezone                              = ""
      # Warning: this attribute value will be marked as sensitive and will
      # not display in UI output after applying this change
      ~ username                              = (sensitive)
        # (41 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Because you flagged the new variables as sensitive, Terraform redacts their values from its output when you run a plan, apply, or destroy command. Notice that the password is marked sensitive value, while the username is marked sensitive. The AWS provider considers the password argument for any database instance as sensitive, whether or not you declare the variable as sensitive, and will redact it as a sensitive value. You should still declare this variable as sensitive to make sure it's redacted if you reference it in other locations than the specific password argument.

Setting values with a .tfvars file allows you to separate sensitive values from the rest of your variable values, and makes it clear to people working with your configuration which values are sensitive. However, it requires that you maintain and share the secret.tfvars file with only the appropriate people. You must also be careful not to check .tfvars files with sensitive values into version control. For this reason, GitHub's recommended .gitignore file for Terraform configuration is configured to ignore files matching the pattern *.tfvars.

Set values with variables

Set the database administrator username and password using environment variables for Terraform Community Edition or Terraform variables for HCP Terraform.

$ export TF_VAR_db_username=admin TF_VAR_db_password=adifferentpassword

$ $Env:TF_VAR_db_username = "admin"; $Env:TF_VAR_db_password = "adifferentpassword"

$ set "TF_VAR_db_username=admin" & set "TF_VAR_db_password=adifferentpassword"

Now, run terraform apply, and Terraform will assign these values to your new variables.

$ terraform apply

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_db_instance.database will be updated in-place
  ~ resource "aws_db_instance" "database" {
        id                                    = "terraform-20220725200214442200000004"
        name                                  = ""
      ~ password                              = (sensitive value)
        tags                                  = {}
        # (57 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.


Do you want to perform these actions in workspace "learn-terraform-sensitive-variables"?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

##...

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.

Respond to the confirmation prompt with yes.

Reference sensitive variables

When you use sensitive variables in your Terraform configuration, you can use them as you would any other variable. Terraform will redact these values in command output and log files, and raise an error when it detects that they will be exposed in other ways.

Add the following output values to outputs.tf.

output "db_connect_string" {
  description = "MySQL database connection string"
  value       = "Server=${aws_db_instance.database.address}; Database=ExampleDB; Uid=${var.db_username}; Pwd=${var.db_password}"
}

Now apply this change. Terraform will raise an error, since the output is derived from sensitive variables.

$ terraform apply
random_string.lb_id: Refreshing state... [id=2Mw]
module.vpc.aws_vpc.this[0]: Refreshing state... [id=vpc-05f973211a47fb6f4]
module.vpc.aws_internet_gateway.this[0]: Refreshing state... [id=igw-0678406e38f54eb60]

## ...

Error: Output refers to sensitive values

  on outputs.tf line 3:
   3: output "db_connect_string" {

Expressions used in outputs can only refer to sensitive values if the
sensitive attribute is true.

Flag the database connection string output as sensitive, causing Terraform to hide it.

output "db_connect_string" {
  description = "MySQL database connection string"
  value       = "Server=${aws_db_instance.database.address}; Database=ExampleDB; Uid=${var.db_username}; Pwd=${var.db_password}"
+  sensitive   = true
}

Apply this change to see that Terraform will now redact the database connection string output. Respond to the confirmation prompt with yes to apply these changes.

$ terraform apply
random_string.lb_id: Refreshing state... [id=2Mw]
module.vpc.aws_vpc.this[0]: Refreshing state... [id=vpc-05f973211a47fb6f4]

## ...

Plan: 0 to add, 0 to change, 0 to destroy.

Changes to Outputs:
  + db_connect_string = (sensitive value)

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

db_connect_string = <sensitive>

Sensitive values in state

When you run Terraform commands with a local state file, Terraform stores the state as plain text, including variable values, even if you have flagged them as sensitive. Terraform needs to store these values in your state so that it can tell if you have changed them since the last time you applied your configuration.

$ grep "password" terraform.tfstate
      "value": "Server=terraform-20210113192204255400000004.ct4cer62f3td.us-east-1.rds.amazonaws.com; Database=ExampleDB; Uid=admin; Pwd=adifferentpassword",
            "password": "adifferentpassword",
## ...

Marking variables as sensitive is not sufficient to secure them. You must also keep them secure while passing them into Terraform configuration, and protect them in your state file. HCP Terraform and Terraform Enterprise manage and share sensitive values, and encrypt all variable values before storing them. HashiCorp Vault secures, stores, and tightly controls access to tokens, passwords, and other sensitive values.

Clean up your infrastructure

Before moving on, destroy the infrastructure you created in this tutorial.

$ terraform destroy

Be sure to respond to the confirmation prompt with yes.

If you used HCP Terraform for this tutorial, after destroying your resources, delete the learn-terraform-sensitive-variables workspace from your HCP Terraform organization.

Next steps

In this tutorial you used Terraform variables to set sensitive information about your infrastructure. In addition to setting the sensitive flag for sensitive inputs, you must also ensure that sensitive values are not accidentally exposed when you pass them into your Terraform configuration. Also, you must keep your Terraform state file secure to avoid accidentally exposing sensitive data.

Now that you know how to use sensitive input variables, check out the following resources for more information.

• Inject secrets into Terraform using the Vault provider by following another tutorial.
• Automatically encrypt state at rest by storing it in HCP Terraform. Get Started with Terraform Cloud.
• Read the documentation about how to manage sensitive data in state.