Authenticate providers with Vault-backed dynamic credentials
You can use HashiCorp Vault to authenticate infrastructure provisioning in HCP Terraform using short-lived, strictly-scoped credentials. These Vault-backed dynamic credentials prevent you from having to store long-lived cloud provider credentials in HCP Terraform. Once you establish a trust relationship between HCP Terraform and Vault, you can use Vault's secrets engine to manage credentials for your cloud provider. HCP Terraform will then request per-operation credentials from Vault to provision your infrastructure.
Vault-backed dynamic credentials are unique and short-lived for each Terraform run, which limits the blast radius in the event of a security breach. Dynamic credentials also give you fine-grained control over the resources that each of your HCP Terraform projects and workspaces can manage by using your cloud providers' access management controls. You can use dynamic credentials to provision infrastructure on cloud providers including AWS, Google Cloud Platform, Azure, and Vault. Vault-backed dynamic credentials let you consolidate your secrets management with Vault and take advantage of its security controls and features such as integration with identity providers.
After you configure Vault-backed dynamic credentials for a workspace or project, HCP Terraform begins each run by requesting credentials from Vault, passing details about the workload, including your organization and workspace name. Vault then authenticates with your cloud provider, and sends the temporary credentials to HCP Terraform. HCP Terraform uses those credentials to manage your resources for the run. This workflow is based on the OpenID Connect protocol (OIDC), an open source standard for verifying identity across different systems.
In this tutorial, you will set up a Vault secrets engine for AWS, establish the trust relationship between Vault and HCP Terraform, and configure a workspace to use Vault to provision dynamic credentials. Finally, you will use dynamic credentials to provision infrastructure in that workspace.
If you do not currently use Vault to manage your credentials, you can configure HCP Terraform to use dynamic provider credentials directly with your cloud provider instead. Refer to the Authenticate Providers with Dynamic Credentials tutorial for details.
Prerequisites
This tutorial assumes that you are familiar with the Terraform, HCP Terraform, and Vault workflows. If you are new to Terraform, complete the Get Started collection first. If you are new to HCP Terraform, complete the HCP Terraform Get Started collection first. If you are new to Vault, complete the Vault Getting Started collection.
For this tutorial, you will need:
- Terraform v1.2+ installed locally.
- An HCP Terraform account and organization.
- A Vault cluster. We recommend you use a non-production HCP Vault cluster to complete this tutorial.
- An AWS account with local credentials configured for use with Terraform.
Clone example repository
Clone the example repository, which contains Terraform configuration to create the trust relationship between AWS and HCP Terraform.
Change into the example repository directory.
Change into the trust subdirectory for AWS.
Configure trust with your cloud provider
To use Vault-backed dynamic credentials, you must first configure Vault's secrets engine, then establish a trust relationship between Vault and HCP Terraform. You must also create the roles and policies that define which resources and services the dynamic credentials can manage, and the HCP Terraform workspace that will provision your resources.
In this tutorial, you will create the trust relationship and TFC workspace by using the CLI-driven workflow for HCP Terraform.
Review cloud provider configuration
Open aws.tf
and review the configuration for your trust relationship.
The configuration first configures the AWS provider, and defines an IAM user that Vault's secrets engine will use to provision your dynamic credentials.
Warning
Terraform will store the credentials that AWS generates for your
aws_iam_access_key.secrets_engine_credentials
resource in your workspace's state.
Always keep your state file secure.
Next, the configuration defines an IAM user policy and role for the
hcp-vault-secrets-engine
user to assume.
Finally, the configuration defines an IAM policy and attaches it to the role. The dynamic credentials that Vault provides will assume the role to provision the resources allowed by the policy.
This policy allows all operations on EC2 instances. IAM policies give you fine-grained control over the permissions granted to your dynamic credentials. You can configure the policies associated with your trust relationships to be as restrictive or permissive as you need.
Review Vault configuration
Open vault.tf
and review the configuration for your trust relationship.
First, the configuration enables Vault's AWS secrets engine and configures it to
authenticate with the IAM user and credentials defined in aws.tf
.
The Vault secrets engine will use these credentials to access the role defined in the previous section.
Next, the configuration configures a JWT authentication backend, which implements the OpenID standard.
This configuration ensures that Vault will check that requests for dynamic credentials satisfy the following OIDC claims:
aud: The intended audience of the token, configured by the
bound_audiences
attribute, matches the unique case-sensitive string for your cloud provider. This ensures that the token is only valid for your configured provider. You may want to customize the audience for advanced use cases, such as using multiple sets of dynamic credentials with a single cloud provider. For this tutorial, use the default value for the audience.sub: The subject of the token, defined in the
bound_claims
attribute, matches the configured HCP Terraform organization name, project, and workspace. This ensures that your cloud provider only authenticates requests for the specified projects and workspaces in your organization, and lets you scope permissions on a per-project or per-workspace basis.
The sub
claim includes four parts separated by colons (:
):
Claim | Value | Purpose |
---|---|---|
organization | ${var.tfc_organization_name} | Your TFC organization name |
project | ${var.tfc_project_name} | Your TFC project name |
workspace | ${var.tfc_workspace_name} | Your TFC workspace name |
run_phase | plan or apply | The Terraform workflow phase |
Tip
Refer to the Dynamic Credentials documentation for a list of other possible claims.
The example configuration uses the bound claims type glob
and the glob
character (*
) to allow any workflow phase. When you define your trust
relationships, you can also use *
to allow requests from any workspace within
a named project, or any project in your organization. Depending on your
organization's needs, you can configure a single trust relationship for each
cloud provider, or separate trust relationships for each project or workspace.
For each run, after Vault verifies that the request is signed by HCP Terraform with the provided TLS certificate, HCP Terraform will provide Vault with a Terraform Workload Identity (TWI) token that includes information about the run, such as the organization, project, and workspace. Vault will then use HCP Terraform's OIDC metadata and public signing key to verify that the token was generated by HCP Terraform and the information it contains has not been tampered with. Once verified, if the token matches the conditions for your trust relationship, Vault will dynamically generate credentials for the configured role. HCP Terraform will then use those dynamic credentials to provision your resources.
Warning
Always include the audience and the name of your HCP Terraform organization when you configure claims. This ensures claims are only valid for your intended cloud provider, and prevents unauthorized access from other HCP Terraform organizations.
Finally, the configuration defines the policy that allows Vault-backed dynamic credentials access to the AWS secrets engine.
Configure HCP Terraform workspace
Open tfe.tf
and review the configuration for your workspace.
First, the configuration defines the TFE provider and an HCP Terraform workspace. The organization and workspace names match those you
defined in the sub
claim for your trust relationship.
Next, the configuration enables Vault-backed dynamic credentials for the workspace by setting the workspace environment variables required to enable and configure the trust relationship.
The TFC_VAULT_PROVIDER_AUTH
variable configures the workspace to use dynamic
credentials for the Vault provider, and the next three variables identify your
Vault instance and the role that Vault will generate dynamic credentials for.
Variable category | Key | Value | Sensitive |
---|---|---|---|
Environment variable | TFC_VAULT_PROVIDER_AUTH | true | No |
Environment variable | TFC_VAULT_ADDR | The address of your Vault instance | No |
Environment variable | TFC_VAULT_RUN_ROLE | vault_run_role output from previous run | No |
Environment variable | TFC_VAULT_NAMESPACE | admin | No |
The next five variables configure the Vault secrets engine for AWS to manage your dynamic credentials and provide them to HCP Terraform.
Variable category | Key | Value | Sensitive |
---|---|---|---|
Environment variable | TFC_VAULT_BACKED_AWS_AUTH | true | No |
Environment variable | TFC_VAULT_BACKED_AWS_MOUNT_PATH | aws | No |
Environment variable | TFC_VAULT_BACKED_AWS_AUTH_TYPE | assumed_role | No |
Environment variable | TFC_VAULT_BACKED_AWS_RUN_ROLE_ARN | ARN of the AWS IAM role | No |
Environment variable | TFC_VAULT_BACKED_AWS_RUN_VAULT_ROLE | Name of the Vault role | No |
Configure organization and workspace name
In the trust
directory for your cloud provider, copy the example .tfvars
file to create a new variables file that configures your organization and workspace name.
Open terraform.tfvars
and configure the variables for the trust relationship.
The configuration will use the organization and workspace name to define the
OIDC subject for your trust relationship's policy.
Replace <YOUR_ORG>
with your HCP Terraform organization name, and
<YOUR_VAULT_CLUSTER>
with your Vault cluster's public URL. If you are using
HCP Vault to complete this tutorial, navigate to your cluster, find the
Cluster URLs tool, and click Public to copy it to your clipboard.
Initialize configuration
Set the VAULT_TOKEN
environment variable to authenticate with your Vault
cluster. If you are using HCP Vault, navigate to your cluster, find the New
admin token tool and click Generate token to copy the token to your
clipboard.
Initialize your Terraform configuration.
Apply configuration
Apply this configuration to create your trust relationship. Respond to the
confirmation prompt with a yes
.
HCP Terraform can now use this trust relationship to request dynamic
credentials from Vault when it provisions infrastructure for the
learn-terraform-dynamic-credentials
workspace in your default project.
Provision infrastructure with dynamic credentials
Now you will use HCP Terraform to provision infrastructure with Vault-backed dynamic credentials using the trust relationship and workspace you provisioned in the last section.
First, navigate to the infra
directory and review the example configuration.
This configuration defines a single EC2 instance in the us-east-2
region.
Initialize configuration
Set the TF_CLOUD_ORGANIZATION
environment variable to your HCP Terraform
organization name. This will configure your HCP Terraform integration.
Now, initialize the configuration.
Apply configuration
In the last section, you configured the trust relationship and the
vault-backed-aws-auth
workspace with the required variables to use it. Your
Vault cluster will trust requests from app.terraform.io
that use your
organization and the vault-backed-aws-auth
workspace.
Apply the configuration to provision your infrastructure using the dynamic credentials.
HCP Terraform will request dynamic credentials from Vault, and use them to
perform a speculative plan. Once the plan is complete, respond to the
confirmation prompt with a yes
to apply your configuration. HCP Terraform
will request another set of dynamic credentials and use them for the apply step.
You successfully provisioned infrastructure with HCP Terraform using dynamic credentials instead of long-lived credentials. Since you do not need to store the credentials themselves in HCP Terraform, this reduces the potential for accidental exposure. Defining the role and permissions for the dynamic credentials lets you specify exactly which operations can use these keys. By using Vault-backed dynamic credentials, you can consolidate secrets management in Vault, and take advantage of its features and integrations.
In your organization, different teams may be responsible for managing trust relationships and deploying infrastructure. For example, a security team could configure and manage the workspaces that define trust relationships, while infrastructure teams configure and manage downstream workspaces that use the trust relationships from the security team. When different teams manage trust and infrastructure, consider how you will coordinate changes to maintain scoped trust relationships while still enabling the downstream teams that rely on them.
Remember that trust relationships reference your organization, project, and workspace names, so changing any of those values within HCP Terraform can potentially break your trust relationships.
Clean up your infrastructure
Remove the infrastructure and trust relationship that you created in this tutorial.
Destroy infrastructure
First, remove the infrastructure you created in this tutorial. In the
aws/infra
directory run terraform destroy
, which will also request dynamic
credentials from Vault to remove your instance.
Destroy trust relationship
Destroy the trust relationship you created in this tutorial.
Navigate to the trust
directory.
Use Terraform to destroy the trust relationship, Vault secret engine
configuration, and workspace. Respond to the confirmation prompt with a yes
.
Next steps
In this tutorial, you created a trust relationship between HCP Terraform and Vault. You also configured Vault's secrets engine to use the trust relationship to provide credentials for your cloud provider. Then you used those credentials to provision your infrastructure.
Refer to the following resources to learn more about dynamic credentials.
Refer to the terraform-dynamic-credentials-setup-examples GitHub repository for other example configurations.
Read the dynamic credentials documentation for more details about how to configure your trust relationships.
You can also manually generate custom auth tokens for custom authentication workflows or providers that do not natively support dynamic credentials.