Vault Agent and Vault Proxy quick start

What is Vault Agent?

Vault Agent behaves as a client-side daemon to make requests to Vault on behalf of the client application. This includes the authentication to Vault.

Vault clients (human users, applications, etc.) must authenticate with Vault and get a client token to make API requests. Because tokens have time-to-live (TTL), the clients must renew the token's TTL or re-authenticate to Vault based on its TTL. Vault Agent authenticates with Vault and manages the token's lifecycle so that the client application doesn't have to.

In addition, you can inject Consul Template markup into Vault Agent so that secrets can be rendered to files where the client application loads data from.

This eliminates the need to change your application code to invoke the Vault API. Your existing applications can remain Vault-unaware.

Prerequisites

To complete this tutorial, you need to install a Vault binary.

Launch Terminal

This tutorial includes a free interactive command-line lab that lets you follow along on actual cloud infrastructure.

Start interactive lab

Lab setup

1. Open a terminal and start a Vault dev server with root as the root token value.

   $ vault server -dev -dev-root-token-id root
   

   Copy

   The Vault dev server defaults to running at 127.0.0.1:8200. The server is also initialized and unsealed.

   Insecure operation: Do not run a Vault dev server in production. This approach is only used here to simplify the unsealing process for this demonstration.

2. Export an environment variable for the vault CLI to address the Vault server.

   $ export VAULT_ADDR=http://127.0.0.1:8200
   

   Copy

3. Log into Vault.

   $ vault login root
   
   Success! You are now authenticated. The token information displayed below
   is already stored in the token helper. You do NOT need to run "vault login"
   again. Future Vault requests will automatically use this token.
   
   Key                  Value
   ---                  -----
   token                root
   token_accessor       9v9YS37o8lvXELvEQY6NqTtV
   token_duration       ∞
   token_renewable      false
   token_policies       ["root"]
   identity_policies    []
   policies             ["root"]
   

   Copy

   Tip

   Vault server is ready and the token value (root) was written to the file $HOME/.vault-token.

Create a working directory

1. Create a directory where you generate test files.

   $ mkdir $HOME/vault-test && cd $HOME/vault-test
   

   Copy

   You will read the created secrets using Vault Agent.

Create test data

1. Create a mock data file.

   $ tee data.json -<<EOF
   {
      "organization": "ACME Inc.",
      "customer_id": "ABXX2398YZPIE7391",
      "region": "US-West",
      "zip_code": "94105",
      "type": "premium",
      "contact_email": "james@acme.com",
      "status": "active"
   }
   EOF
   

   Copy

2. Create the mock data in the KV v2 secrets engine.

   $ vault kv put secret/customers/acme @data.json
   
   ======= Secret Path =======
   secret/data/customers/acme
   
   ======= Metadata =======
   Key                Value
   ---                -----
   created_time       2023-02-23T16:25:46.030960819Z
   custom_metadata    <nil>
   deletion_time      n/a
   destroyed          false
   version            1
   

   Copy

Start a Vault Agent

Start a Vault Agent instance that connects to the Vault server running at VAULT_ADDR.

1. Create the Vault Agent configuration file.

   $ tee agent-config.hcl -<<EOF
   pid_file = "./pidfile"
   
   vault {
      address = "$VAULT_ADDR"
      tls_skip_verify = true
   }
   
   auto_auth {
      method {
         type = "token_file"
         config = {
            token_file_path = "$HOME/.vault-token"
         }
      }
      sink "file" {
         config = {
               path = "$HOME/vault-token-via-agent"
         }
      }
   }
   EOF
   

   Copy

   Note

   As of Vault 1.13.0, you can use the client token stored in a file (token_file_path). This is a convenient way to authenticate the Vault Agent with Vault server during development. For production deployment, use one of the supported auth methods.

2. Start a Vault Agent.

   $ vault agent -config=agent-config.hcl
   

   Copy

   Example output:

   ==> Vault Agent started! Log data will stream in below:
   
   ==> Vault Agent configuration:
   
                        Cgo: disabled
                  Log Level:
                  Version: Vault v1.13.0, built 2023-02-14T14:25:23Z
               Version Sha: c2d6b060c0eec60393e866b761a5990716cb45de
   
   [INFO]  vault-agent.sink.file: creating file sink
   [INFO]  vault-agent.sink.file: file sink configured: path=$HOME/vault-token-via-agent mode=-rw-r-----
   [INFO]  vault-agent.template.server: starting template server
   [INFO]  vault-agent.template.server: no templates found
   [INFO]  vault-agent.auth.handler: starting auth handler
   [INFO]  vault-agent.auth.handler: authenticating
   [INFO]  vault-agent.sink.server: starting sink server
   [INFO]  vault-agent.auth.handler: authentication successful, sending token to sinks
   ...snip...
   

At this point, you still need to change your client application to make Vault requests targeting VAULT_ADDR using the Vault token stored in the sink location ($HOME/vault-token-via-agent).

Next, explore other Vault Agent features.

Modify the Vault Agent configuration

Starting with Vault 1.13.0, you can specify multiple Vault Agent configuration files.

$ vault agent -config=file-1.hcl -config=file-2.hcl

Or, point to a directory with the configuration files.

$ vault agent -config=directory/

In this section, you are going to define additional Vault Agent configurations.

Define Vault Agent template

In the following scenario, the client application loads data read from customer.json.

1. Press Ctrl + C to stop the running Vault Agent.

2. Create a customer.json.tmpl file.

   $ tee customer.json.tmpl -<<EOF
   {
      {{ with secret "secret/data/customers/acme" }}
      "Organization": "{{ .Data.data.organization }}",
      "ID": "{{ .Data.data.customer_id }}",
      "Contact": "{{ .Data.data.contact_email }}"
      {{ end }}
   }
   EOF
   

   Copy

3. Create a Vault Agent configuration file with a template stanza.

   $ tee agent-template.hcl -<<EOF
   template {
      source      = "$HOME/vault-test/customer.json.tmpl"
      destination = "$HOME/vault-test/customer.json"
   }
   EOF
   

   Copy

4. Start Vault Agent again with the additional configuration applied.

   $ vault agent -config=agent-config.hcl \
      -config=agent-template.hcl
   

   Copy

   Example output:

   ==> Vault Agent started! Log data will stream in below:
   
   ==> Vault Agent configuration:
   
            Api Address 1: http://bufconn
                        Cgo: disabled
                  Log Level:
                  Version: Vault v1.13.0-rc1+ent, built 2023-02-14T14:25:23Z
               Version Sha: c2d6b060c0eec60393e866b761a5990716cb45de
   
   [INFO]  vault-agent.sink.file: creating file sink
   [INFO]  vault-agent.sink.file: file sink configured: path=$HOME/vault-token-via-agent mode=-rw-r-----
   [INFO]  vault-agent.auth.handler: starting auth handler
   [INFO]  vault-agent.auth.handler: authenticating
   [INFO]  vault-agent.template.server: starting template server
   ...snip...
   [INFO] (runner) starting
   [INFO] (runner) rendered "$HOME/vault-test/customer.json.tmpl" => "$HOME/vault-test/customer.json"
   

5. Open a new browser or text editor and verify that the customer.json file has the secrets retrieved from Vault.

   $ cat customer.json
   

   Copy

   Example output:

      {
   
      "Organization": "ACME Inc.",
      "ID": "ABXX2398YZPIE7391",
      "Contact": "james@acme.com"
   
   }
   

   This should match the data stored in secret/.

   $ vault kv get secret/customers/acme
   
   ======= Secret Path =======
   secret/data/customers/acme
   
   ======= Metadata =======
   Key                Value
   ---                -----
   created_time       2023-02-23T16:25:46.030960819Z
   custom_metadata    <nil>
   deletion_time      n/a
   destroyed          false
   version            1
   
   ======== Data ========
   Key              Value
   ---              -----
   contact_email    james@acme.com
   customer_id      ABXX2398YZPIE7391
   organization     ACME Inc.
   region           US-West
   status           active
   type             premium
   zip_code         94105
   

   Copy
   Troubleshooting for HCP Vault Dedicated

   If the template fails, make sure to set the VAULT_NAMESPACE variable.

   $ echo $VAULT_NAMESPACE
   admin
   

   Copy

The client application was designed to load data from the customer.json file. By leveraging the Vault Agent Template feature, you did not have to make any code changes to the client application while you manage secrets with Vault.

Vault Agent logs

You can specify the behavior of Vault Agent logging using a command flag, environment variable, or configuration parameters.

Vault Proxy

Vault Proxy acts as a proxy between Vault and its client applications. See the Vault Agent and Vault Proxy documentation page for more details about feature parity between Vault Agent and Vault Proxy.

Vault Proxy provides auto-auth, caching, and API proxy functionality.

The vault and auto_auth stanzas in the Vault Proxy and Vault Agent configurations are identical; however, you must define listener and api_proxy stanzas for Vault Proxy.

Example configuration for Vault Proxy:

pid_file = "./pidfile"

vault {
   address = "http://127.0.0.1:8200"
   tls_skip_verify = true
}

auto_auth {
   method {
      type = "token_file"
      config = {
         token_file_path = "$HOME/.vault-token"
      }
   }
   sink "file" {
      config = {
            path = "$HOME/vault-proxy"
      }
   }
}

listener "tcp" {
   address     = "127.0.0.1:8100"
   tls_disable = true
}

api_proxy {
   use_auto_auth_token = true
   enforce_consistency = "always"
}

cache {
   // Requires Vault Enterprise 1.16 or later
   cache_static_secrets = true
   static_secret_token_capability_refresh_interval = "5m"
}

Start a Vault Proxy

You can define an API endpoint for the client application to send requests to rather than VAULT_ADDR.

1. Press Ctrl + C to stop the running Vault Agent.

2. Create a file that defines the listener stanza and api_proxy stanza.

   $ tee vault-proxy-config.hcl -<<EOF
   pid_file = "./pidfile"
   
   vault {
      address = "$VAULT_ADDR"
      tls_skip_verify = true
   }
   
   auto_auth {
      method {
         type = "token_file"
         config = {
            token_file_path = "$HOME/.vault-token"
         }
      }
      sink "file" {
         config = {
               path = "$HOME/vault-token-via-agent"
         }
      }
   }
   
   listener "tcp" {
      address     = "127.0.0.1:8100"
      tls_disable = true
   }
   
   api_proxy {
      use_auto_auth_token = true
      enforce_consistency = "always"
   }
   
   cache {
      // Requires Vault Enterprise 1.16 or later
      cache_static_secrets = true
      static_secret_token_capability_refresh_interval = "5m"
   }
   EOF
   

   Copy

   This configuration creates a listener endpoint on the loopback interface 127.0.0.1:8100. You can add multiple listener stanzas. See the Vault Proxy documentation for more information.

   The api_proxy stanza enables the use_auto_auth_token parameter. When enabled, proxied requests do not need to set the X-Vault-Token HTTP header, as the auto-auth token will be used by default.

   Vault Enterprise 1.16 or later

   Set the cache_static_secrets parameter in the cache stanza to true to enable the static secrets caching feature that requires Vault Enterprise 1.16 or later. This enables Vault Proxy to cache static secrets read from the KV v1 and KV v2 secrets engines.

   Note that the persistent caching feature available in Vault Community Edition is currently supported only in Kubernetes environments and does not cache static secrets.

3. Start the Vault Proxy.

   $ vault proxy -config=vault-proxy-config.hcl
   

   Copy

   Vault Proxy is now listening to the proxy addresses http://127.0.0.1:8100.

   ==> Vault Proxy started! Log data will stream in below:
   
   ==> Vault Proxy configuration:
   
            Api Address 1: http://127.0.0.1:8100
                        Cgo: disabled
                  Log Level:
                  Version: Vault v1.16.0, built 2024-02-12T16:07:06Z
               Version Sha: 2ca8655bfeb501c0937240df2828ef0fb7c78de1
   

4. Open another command terminal and send an API request to the Vault Proxy.

   Read the secrets at secret/customers/acme via the proxy address.

   This example uses jq to process the JSON output for readability.

   $ curl -s http://127.0.0.1:8100/v1/secret/data/customers/acme \
      | jq -r ".data.data"
   

   Copy

   Output:

   {
      "contact_email": "james@acme.com",
      "customer_id": "ABXX2398YZPIE7391",
      "organization": "ACME Inc.",
      "region": "US-West",
      "status": "active",
      "type": "premium",
      "zip_code": "94105"
   }
   

   Although no client token was provided in the API request (X-Vault-Token header unset), a token was automatically attached to the request by the Vault Proxy because the use_auto_auth_token parameter was set to true in the configuration.

Static secret caching

1. Read the secrets again. This time, Vault Proxy returns the cached secrets.

   $ curl --verbose -s http://127.0.0.1:8100/v1/secret/data/customers/acme \
      | jq -r ".data.data"
   

   Copy

   Use the --verbose or -v option with the cURL command so that you can see more detail.

   *   Trying 127.0.0.1:8100...
   * Connected to 127.0.0.1 (127.0.0.1) port 8100
   > GET /v1/secret/data/customers/acme HTTP/1.1
   > Host: 127.0.0.1:8100
   > User-Agent: curl/8.4.0
   > Accept: */*
   >
   < HTTP/1.1 200 OK
   < Age: 65
   < Cache-Control: no-store
   < Content-Length: 483
   < Content-Type: application/json
   < Date: Tue, 20 Feb 2024 19:28:17 GMT
   < Strict-Transport-Security: max-age=31536000; includeSubDomains
   < X-Cache: HIT
   <
   { [483 bytes data]
   * Connection #0 to host 127.0.0.1 left intact
   {
      "contact_email": "james@acme.com",
      "customer_id": "ABXX2398YZPIE7391",
      "organization": "ACME Inc.",
      "region": "US-West",
      "status": "active",
      "type": "premium",
      "zip_code": "94105"
   }
   

   Notice that the X-Cache is HIT.

2. Update the secrets to see what happens.

   $ vault kv patch secret/customers/acme "customer_since"="2011"
   

   Copy

3. Read the secrets again via proxy.

   $ curl --verbose http://127.0.0.1:8100/v1/secret/data/customers/acme \
      | jq -r ".data.data"
   

   Copy

   The Age value changed since the secrets were just updated.

   *   Trying 127.0.0.1:8100...
   * Connected to 127.0.0.1 (127.0.0.1) port 8100
   > GET /v1/secret/data/customers/acme HTTP/1.1
   > Host: 127.0.0.1:8100
   > User-Agent: curl/8.4.0
   > Accept: */*
   >
   < HTTP/1.1 200 OK
   < Age: 8
   < Cache-Control: no-store
   < Content-Length: 506
   < Content-Type: application/json
   < Date: Tue, 20 Feb 2024 19:29:21 GMT
   < Strict-Transport-Security: max-age=31536000; includeSubDomains
   < X-Cache: HIT
   <
   { [506 bytes data]
   * Connection #0 to host 127.0.0.1 left intact
   {
      "contact_email": "james@acme.com",
      "customer_id": "ABXX2398YZPIE7391",
      "customer_since": "2011",
      "organization": "ACME Inc.",
      "region": "US-West",
      "status": "active",
      "type": "premium",
      "zip_code": "94105"
   }
   

4. Press Ctrl + C to stop the running Vault Proxy.

Self-healing tokens

Auto-auth authenticates with Vault to get a client token, store, and manage the token lifecycle. Vault 1.17 enhanced this auto-auth capability to increase its resiliency so that Vault Agent and Vault Proxy can heal from an invalid token error caused by unexpected incidence (for example, Vault admin revoked the client token used by the Vault Agent).

Test the self-healing capability

1. Create a script to setup approle auth method.

   $ tee setup-approle.sh -<<EOF
   ## Create a policy named "read-only"
   vault policy write read-only -<<EOL
   path "secret/*" {
      capabilities = ["read", "list"]
   }
   EOL
   
   ## Enable and configure approle auth method
   vault auth enable approle
   vault write auth/approle/role/read-only policies=read-only token_num_uses=5
   
   ## Retrive role ID and secret ID
   vault read -field=role_id auth/approle/role/read-only/role-id > $HOME/vault-test/roleID.txt
   vault write -f -field=secret_id auth/approle/role/read-only/secret-id > $HOME/vault-test/secretID.txt
   EOF
   

   Copy

   This script:

   • Creates a policy named "read-only" which allows read and list operations against paths starts with secret/.
   • Enables approle auth method.
   • Creates a role named "read-only" which has the read-only policy attached. The token for this role has a use limit of 5.
   • Retrives the role ID for the "read-only" role and store the value in the roleID.txt file.
   • Retrieves the secret ID and store the value in the secretID.txt file.

2. Make the script executable.

   $ chmod +x setup-approle.sh
   

   Copy

3. Run the script to setup an approle auth method.

   $ ./setup-approle.sh
   
   Success! Uploaded policy: read-only
   Success! Enabled approle auth method at: approle/
   Success! Data written to: auth/approle/role/read-only
   

   Copy

4. Create a new Vault Proxy configuration which uses approle auth method.

   $ tee vault-proxy-config-2.hcl -<<EOF
   pid_file = "./pidfile"
   vault {
      address = "$VAULT_ADDR"
      tls_skip_verify = true
   }
   
   auto_auth {
      method {
         type = "approle"
         config = {
            role_id_file_path = "$HOME/vault-test/roleID.txt"
            secret_id_file_path = "$HOME/vault-test/secretID.txt"
         }
      }
      sink "file" {
         config = {
               path = "$HOME/vault-token-via-agent"
         }
      }
   }
   
   listener "tcp" {
      address     = "127.0.0.1:8100"
      tls_disable = true
   }
   
   api_proxy {
      use_auto_auth_token = true
      enforce_consistency = "always"
   }
   EOF
   

   Copy

5. Start a Vault Proxy.

   $ vault proxy -config=vault-proxy-config-2.hcl
   

   Copy

6. From another command terminal, read the secrets through the proxy.

   $ curl -s http://127.0.0.1:8100/v1/secret/data/customers/acme \
      | jq -r ".data.data"
   

   Copy

   The command successfully returns the secrets.

   {
      "contact_email": "james@acme.com",
      "customer_id": "ABXX2398YZPIE7391",
      "organization": "ACME Inc.",
      "region": "US-West",
      "status": "active",
      "type": "premium",
      "zip_code": "94105"
   }
   

7. Revoke the token which is stored in the sink location.

   $ vault token revoke $(cat $HOME/vault-token-via-agent)
   Success! Revoked token (if it existed)
   

   Copy

8. Try to read the secrets again.

   $ curl -s http://127.0.0.1:8100/v1/secret/data/customers/acme \
      | jq -r ".data.data"
   

   Copy

   The request fails and returns no data.

9. In the terminal where Vault Proxy is running, check the Vault Proxy logs.

   ...snip...
   [INFO]  proxy.apiproxy: proxy received an invalid token error
   [INFO]  proxy.auth.handler: invalid token found, re-authenticating
   [INFO]  proxy.auth.handler: authenticating
   [INFO]  proxy.auth.handler: authentication successful, sending token to sinks
   [INFO]  proxy.auth.handler: starting renewal process
   [INFO]  proxy.sink.file: token written: path=/user/vault-token-via-agent
   [INFO]  proxy.auth.handler: renewed auth token
   

   The log indicates invalid token error because you revoked the client token. The Vault Agent re-authenticate with Vault to get a new token, and stores it in the sink location.

10. Try to read the secret. This time, you should received the secrets.

    $ curl -s http://127.0.0.1:8100/v1/secret/data/customers/acme \
       | jq -r ".data.data"
    

    Copy

Clean up

1. Press Ctrl + C to stop the running Vault Agent.

2. You can stop the Vault dev server by pressing Ctrl+C where the server is running. Or, execute the following command.

   $ pgrep -f vault | xargs kill
   

   Copy

3. Unset the VAULT_ADDR environment variable.

   $ unset VAULT_ADDR
   

   Copy

4. Delete the $HOME/vault-test directory.

   $ cd .. && rm -r $HOME/vault-test
   

   Copy

Summary

This tutorial demonstrated the basics of Vault Agent. You learned how to configure and run Vault Agent and Vault Proxy.

The use of token_file in the auto-auth stanza is a convenient way to quickly start the Vault Agent or Vault Proxy. For a production deployment, use one of the supported auth methods so that the Vault Agent or Vault Proxy can properly manage the token's lifecycle.

See the following tutorials for auto-auth examples:

• Vault Agent with AWS
• Vault Agent with Kubernetes

The Vault Agent Template demonstrated that the client application can remain Vault-unaware while Vault manages the secrets.

See the following tutorials to see more template examples:

• Vault Agent Templates
• Using HashiCorp Vault Agent with .NET Core

Help and reference

• Vault Agent documentation
• Vault Agent Auto-Auth
• Vault Agent Caching