HashiCorp Developer AI Private beta now available Sign up today
Vault
Authentication

Google Cloud Platform (GCP) auth method

  • 15min
  • |
  • Vault

Before a client can interact with Vault, it must authenticate with an auth method to acquire a token. This token has policies attached so that the behavior of the client can be governed.

Auth Concept

NOTE: To learn the basics of Vault tokens, go through the Tokens tutorial.

Auth methods perform authentication to verify the user or machine-supplied information. Some of the supported auth methods are targeted towards users while others are targeted toward machines or apps.

Challenge

Vault supports a number of auth methods for users or system to prove their identity so that a token with appropriate policies can be obtained.

Google Cloud Platform customers require a solution that supports multiple services for both users and compute services.

Solution

The gcp auth method allows Google Cloud Platform entities to authenticate to Vault. Vault treats Google Cloud as a trusted third party and verifies authenticating entities against the Google Cloud APIs.

Prerequisites

This tutorial assumes the following:

  • You have a Google Cloud Platform account with permission to:
    • Create IAM service accounts, policies
    • Enable GCP APIs
    • Access the GCP Cloud Shell
    • Create VMs
  • Vault installed on your local machine
  • ngrok installed and configured with an auth token (required for Vault)

Lab setup

Workstation setup

  1. Create a temporary directory to store the files used for this tutorial.

    $ mkdir ~/TUTORIAL_TEMP

  2. Retrieve the directory path and store it in the TUTORIAL_TEMP environment variable.

    $ TUTORIAL_TEMP=$(echo ~/TUTORIAL_TEMP)

  3. Go to the TUTORIAL_TEMP directory.

    $ cd $TUTORIAL_TEMP

Vault setup

Note

If you do not have access to an HCP Vault Dedicated cluster, visit the Create a Vault Cluster on HCP tutorial.

  1. Launch the HCP Portal and login.

  2. Click Vault in the left navigation pane.

  3. In the Vault clusters pane, click vault-cluster.

  4. Under Cluster URLs, click Public Cluster URL. Public Cluster URL

  5. In the same terminal you set the TUTORIAL_TEMP environment variable, set the VAULT_ADDR environment variable to the copied address.

    $ export VAULT_ADDR=<Public_Cluster_URL>

  6. Return to the Overview page and click Generate token. Generate a Token

    Within a few moments, a new token will be generated.

  7. Copy the Admin Token. Generated Token

  8. Return to the terminal and set the VAULT_TOKEN environment variable.

    $ export VAULT_TOKEN=<token>

  9. Set the VAULT_NAMESPACE environment variable to admin.

    $ export VAULT_NAMESPACE=admin

    The admin namespace is the top-level namespace automatically created by HCP Vault. All CLI operations default to use the namespace defined in this environment variable.

    Note

    For these tasks, you can use HCP Vault Dedicated's admin token. However, it is recommended that admin tokens are only used for enough initial setup or in emergencies. As a best practice, use an authentication method or token that meets the policy requirements.

  10. Enable the KV secrets engine.

    $ vault secrets enable -version=2 -path=secret kv

Success! Enabled the kv secrets engine at: secret/

    The Vault Dedicated server is ready.

Configure GCP services for Vault

(Persona: admin)

Warning

Resources will be provisioned during this tutorial that may result in charges to your Google Cloud Platform account.

Before Vault can authenticate GCP IAM service accounts and GCE instances, you need to configure the necessary resources in GCP. This includes enabling the required GCP APIs, creating a IAM service account and IAM policy for Vault, and creating a key for the service account that Vault will use to authenticate with GCP.

  1. Open a browser and sign into the GCP console.

  2. Click the terminal icon to launch the GCP Cloud Shell.

    A new pane will open at the bottom of your window and provision a Cloud Shell instance.

  3. In the Cloud Shell terminal, list the GCP project ID.

    Note

    If this is the first time using the Cloud Shell terminal, you will be prompted to authorize permission to use your credentials.

    Click Authorize when prompted.

    $ gcloud projects list

PROJECT_ID: abc-123defde860044c59e853ee2819
NAME: my-gcp-account-name
PROJECT_NUMBER: 85527753462

    The project ID is used when configuring both GCP and Vault.

  4. Verify the cloud resource manager and IAM API's are enabled.

    $ gcloud services list --enabled | grep 'resource\|iam\|compute'

NAME: cloudresourcemanager.googleapis.com
NAME: compute.googleapis.com
NAME: iam.googleapis.com
NAME: iamcredentials.googleapis.com

    If cloudresourcemanager.googleapis.com, compute.googleapis.com, or iam.googleapis.com are not listed, expand the menu below and enable the required API(s).

    Enable the iam.googleapis.com API.

    $ gcloud services enable iam.googleapis.com

    Enable the compute.googleapis.com API.

    $ gcloud services enable compute.googleapis.com

    Enable the cloudresourcemanager.googleapis.com API.

    $ gcloud services enable cloudresourcemanager.googleapis.com

  5. Create a service account for Vault to authenticate with GCP.

    $ gcloud iam service-accounts create \
    VaultServiceAccount \
    --display-name="VaultServiceAccount"

  6. Create a GCP IAM role for Vault and assign the required permissions.

    Note

    The permissions assigned in this tutorial are for both IAM and GCE.

    If you only require one of these services, remove the unused permissions.

    $ gcloud iam roles create VaultServiceRole \
    --project=$DEVSHELL_PROJECT_ID \
    --title=VaultServiceRole \
    --stage=GA \
    --permissions=iam.serviceAccounts.get,iam.serviceAccountKeys.get,compute.instances.get,compute.instanceGroups.list,iam.serviceAccounts.signJwt

    Note

    The iam.serviceAccounts.signJwt permission is required for any service account used to authenticate with Vault using the IAM method. If you are not using the IAM method, you can remove this permission.

  7. Retrieve the role name and store it in the ROLE_NAME environment variable.

    $ ROLE_NAME=$(gcloud iam roles list --project=$DEVSHELL_PROJECT_ID --format=json --filter="vault" | jq -r .[].name)

  8. Retrieve the service account email and store it in the SERVICE_ACCOUNT environment variable.

    $ SERVICE_ACCOUNT=$(gcloud iam service-accounts list --project=$DEVSHELL_PROJECT_ID --filter=vault --format=json | jq -r .[].email)

  9. Add the Vault service account to the GCP project and bind the role to the service account.

    $ gcloud projects add-iam-policy-binding $DEVSHELL_PROJECT_ID \
    --member="serviceAccount:$SERVICE_ACCOUNT" \
    --role="$ROLE_NAME"

  10. Create a service account key credential file.

    $ gcloud iam service-accounts keys create VaultServiceAccountKey.json \
    --iam-account=$SERVICE_ACCOUNT \
    --project=$DEVSHELL_PROJECT_ID

  11. Click the More button (represented by the vertical ellipsis) and select Download.

  12. Click the folder icon, expand the directory, select the VaultServiceAccountKey.json file, and click Download.

  13. Store the VaultServiceAccountKey.json file in TUTORIAL_TEMP directory you created in the lab setup section.

    Tip

    Depending on your browser settings, the file may automatically download to a default directory. To complete this tutorial, make sure the VaultServiceAccountKey.json file is located in the $TUTORIAL_TEMP directory.

    Example OSX command:

    $ mv ~/Downloads/VaultServiceAccountKey.json $TUTORIAL_TEMP/

  14. Retrieve and copy the GCP project ID from the Cloud Shell terminal.

    $ echo $DEVSHELL_PROJECT_ID

  15. Return to the terminal where you set the TUTORIAL_TEMP environment variable and set the GCP_PROJECT_ID environment variable to the value from the previous step.

    $ export GCP_PROJECT_ID=<actual-project-id>

    You have completed the necessary configuration in GCP to support the Vault GCP secrets engine.

Configure Vault

(Persona: admin)

Note

The GCP auth method focuses on identities specific to Google Cloud and does not support authenticating arbitrary Google or Google Workspace users or generic OAuth against Google.

  1. In the terminal where you set the TUTORIAL_TEMP environment variable, enable the GCP secrets engine.

    $ vault auth enable gcp

Success! Enabled gcp auth method at: gcp/

  2. Configure the GCP auth method to use the VaultServiceAccountKey.json credentials.

    $ vault write auth/gcp/config \
    credentials=@$TUTORIAL_TEMP/VaultServiceAccountKey.json

    Example output:

    Success! Data written to: auth/gcp/config

  3. Create a policy file named dev.hcl.

    $ tee dev.hcl <<EOF
  # Read permission on the k/v secrets
  path "/secret/*" {
      capabilities = ["read", "list"]
  }
EOF

  4. Create a policy named dev with the policy defined in dev.hcl.

    $ vault policy write dev dev.hcl

Success! Uploaded policy: dev

  5. Create a role for GCE instances.

    A role includes several parameters.

    • policies: One or more comma separated Vault policies that will be applied.
    • bound_projects: The name of your GCP project.
    • bound_zones: One or more comma separated GCP zones permitted to authenticate with Vault.
    $ vault write auth/gcp/role/vault-gce-auth-role \
   type="gce" \
   policies="dev" \
   bound_projects="$GCP_PROJECT_ID" \
   bound_zones="us-east1-b" \

    Review the GCP auth method API documentation for a full list of supported parameters.

    The Vault configuration is complete

Authenticate with Vault from GCP

(Persona: app)

To complete authentication with Vault, a signed JSON Web Token must be passed to Vault. The Vault binary simplifies this process. Refer to the generating JWTs documentation for steps on retrieving the JWT manually.

Deploy GCE VM

  1. Return to the GCP console.

  2. Expand the hamburger menu and navigate to Compute Engine >> VM instances.

  3. Click CREATE INSTANCE with the following details:

    • Name: vault-auth-test
    • Region: us-east1
    • Zone: us-east1-b

  4. Keep all other options set to their default values and click Create.

  5. When the instance becomes available, click SSH. A new window will open and log you into the VM.

Setup Vault CLI

The Vault binary will require access to your Vault Dedicated cluster. Ensure that you followed the Lab setup section and enabled public access. In production environments, you can connect your GCP account via an AWS transit gateway and use the Vault Dedicated private address.

  1. Download the HashiCorp GPG key.

    $ curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg

  2. Add the HashiCorp repo.

    $ echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list

  3. Install Vault.

    $ sudo apt update && sudo apt install vault

  4. Return to the HCP Portal and login if necessary.

  5. Click Vault in the left navigation pane.

  6. In the Vault clusters pane, click vault-cluster.

  7. Under Cluster URLs, click Public Cluster URL. Public Cluster URL

  8. Return to the SSH session for the GCP instance and set the VAULT_ADDR environment variable to the copied address.

    $ export VAULT_ADDR=<Public_Cluster_URL>

  9. Set the VAULT_NAMESPACE environment variable to admin.

    $ export VAULT_NAMESPACE=admin

  10. Authenticate with Vault using the vault-gce-auth-role role.

    $ vault login -method=gcp role="vault-gce-auth-role"

Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.

Key                                       Value
---                                       -----
token                                     hvs.CAESIJr5Wfq_LO0TOVb4CKqkHq1bJGeSkEX5dvdm8f6QM8MBGicKImh2cy4zTTlNaE1iZFV0T0pVZFNreG43TUhZWjAuQ0xHemgQsQY
token_accessor                            V0eS8zohjaTBnLDQEucMsJKM.CLGzh
token_duration                            1h
token_renewable                           true
token_policies                            ["default" "dev"]
identity_policies                         []
policies                                  ["default" "dev"]
token_meta_project_number                 1060482098095
token_meta_zone                           us-east1-b
token_meta_instance_creation_timestamp    1678121533
token_meta_instance_id                    1617931539497854164
token_meta_project_id                     hc-c0af9a5c6c9a42328ef9b7ce806
token_meta_service_account_id             115056664069215052875
token_meta_instance_name                  instance-1
token_meta_role                           vault-gce-auth-role
token_meta_service_account_email          1060482098095-compute@developer.gserviceaccount.com

    You have successfully authenticated with Vault using the GCP auth method from a GCE instance.

    The dev policy was assigned through the vault-gce-auth-role.

Cleanup

To avoid unnecessary charges and having unused security resources in your GCP account, HCP account, or local workstation, remove the following:

Cleanup Vault

  1. Return to the HCP Portal.

  2. Delete any Vault clusters created for this tutorial.

Cleanup GCP

  1. Return to the GCP console and close the Cloud Shell terminal.

  2. Expand the hamburger menu and navigate to Compute Engine >> VM instances. Delete any GCE instances created for this tutorial.

  3. Expand the hamburger menu and navigate to IAM & Admin >> Service accounts. Delete the VaultServiceAccount service account.

  4. From the IAM & Admin console, click Roles and delete the VaultServiceRole role.

Cleanup local workstation

  1. On your local workstation, switch to the terminal used to configure Vault and create an array of the local environment variables.

    $ learnVariables=( \
GCP_PROJECT_ID \
GCP_SERVICE_EMAIL \
VAULT_ADDR \
VAULT_TOKEN \
VAULT_NAMESPACE
)

  2. Unset the environment variables.

    for v in ${learnVariables[@]}; do
unset $v
done

  3. Remove the TUTORIAL_TEMP directory.

    $ cd ~ && rm -rf $TUTORIAL_TEMP && unset TUTORIAL_TEMP

Help and reference

Previous

OIDC authentication with Google Workspace

 Next

SAML authentication