Vault proxy caching

Vault Proxy Caching allows client-side caching of responses containing newly created tokens and responses containing leased secrets generated off of these newly created tokens. The renewals of the cached tokens and leases are also managed by the proxy.

Caching and renewals

Response caching and renewals are managed by the proxy only under these specific scenarios.

1. Token creation requests are made through the proxy. This means that any login operations performed using various auth methods and invoking the token creation endpoints of the token auth method via the proxy will result in the response getting cached by the proxy. Responses containing new tokens will be cached by the proxy only if the parent token is already being managed by the proxy or if the new token is an orphan token.

2. Leased secret creation requests are made through the proxy using tokens that are already managed by the proxy. This means that any dynamic credentials that are issued using the tokens managed by the proxy, will be cached and its renewals are taken care of.

Persistent cache

Vault Proxy can restore tokens and leases from a persistent cache file created by a previous Vault Proxy process.

Refer to the Vault Proxy Persistent Caching page for more information on this functionality.

Cache evictions

The eviction of cache entries pertaining to secrets will occur when the proxy can no longer renew them. This can happen when the secrets hit their maximum TTL or if the renewals result in errors.

Vault Proxy does some best-effort cache evictions by observing specific request types and response codes. For example, if a token revocation request is made via the proxy and if the forwarded request to the Vault server succeeds, then proxy evicts all the cache entries associated with the revoked token. Similarly, any lease revocation operation will also be intercepted by the proxy and the respective cache entries will be evicted.

Note that while proxy evicts the cache entries upon secret expirations and upon intercepting revocation requests, it is still possible for the proxy to be completely unaware of the revocations that happen through direct client interactions with the Vault server. This could potentially lead to stale cache entries. For managing the stale entries in the cache, an endpoint /proxy/v1/cache-clear(see below) is made available to manually evict cache entries based on some of the query criteria used for indexing the cache entries.

Request uniqueness

In order to detect repeat requests and return cached responses, proxy will need to have a way to uniquely identify the requests. This computation as it stands today takes a simplistic approach (may change in future) of serializing and hashing the HTTP request along with all the headers and the request body. This hash value is then used as an index into the cache to check if the response is readily available. The consequence of this approach is that the hash value for any request will differ if any data in the request is modified. This has the side-effect of resulting in false negatives if say, the ordering of the request parameters are modified. As long as the requests come in without any change, caching behavior should be consistent. Identical requests with differently ordered request values will result in duplicated cache entries. A heuristic assumption that the clients will use consistent mechanisms to make requests, thereby resulting in consistent hash values per request is the idea upon which the caching functionality is built upon.

Renewal management

The tokens and leases are renewed by the proxy using the secret renewer that is made available via the Vault server's Go API. Proxy performs all operations in memory and does not persist anything to storage. This means that when the proxy is shut down, all the renewal operations are immediately terminated and there is no way for the proxy to resume renewals after the fact. Note that shutting down the proxy does not indicate revocations of the secrets, instead it only means that renewal responsibility for all the valid unrevoked secrets are no longer performed by the Vault proxy.

API

Cache clear

This endpoint clears the cache based on given criteria. To use this API, some information on how the proxy caches values should be known beforehand. Each response that is cached in the proxy will be indexed on some factors depending on the type of request. Those factors can be the token that is belonging to the cached response, the token_accessor of the token belonging to the cached response, the request_path that resulted in the cached response, the lease that is attached to the cached response, the namespace to which the cached response belongs to, and a few more. This API exposes some factors through which associated cache entries are fetched and evicted. For listeners without caching enabled, this API will still be available, but will do nothing (there is no cache to clear) and will return a 200 response.

Parameters

• type (strings: required) - The type of cache entries to evict. Valid values are request_path, lease, token, token_accessor, and all. If the type is set to all, the entire cache is cleared.

• value (string: required) - An exact value or the prefix of the value for the type selected. This parameter is optional when the type is set to all.

• namespace (string: optional) - This is only applicable when the type is set to request_path. The namespace of which the cache entries to be evicted for the given request path.

Sample payload

{
  "type": "token",
  "value": "hvs.rlNjegSKykWcplOkwsjd8bP9"
}

Sample request

$ curl \
    --request POST \
    --data @payload.json \
    http://127.0.0.1:1234/proxy/v1/cache-clear

Configuration (cache)

The presence of the top level cache block in any way (including an empty cache block) will enable the cache. The top level cache block has the following configuration entry:

• persist (object: optional) - Configuration for the persistent cache.

Configuration (Persist)

These are common configuration values that live within the persist block:

• type (string: required) - The type of the persistent cache to use, e.g. kubernetes. Note: when using HCL this can be used as the key for the block, e.g. persist "kubernetes" {...}. Currently, only kubernetes is supported.

• path (string: required) - The path on disk where the persistent cache file should be created or restored from.

• keep_after_import (bool: optional) - When set to true, a restored cache file is not deleted. Defaults to false.

• exit_on_err (bool: optional) - When set to true, if any errors occur during a persistent cache restore, Vault Proxy will exit with an error. Defaults to true.

• service_account_token_file (string: optional) - When type is set to kubernetes, this configures the path on disk where the Kubernetes service account token can be found. Defaults to /var/run/secrets/kubernetes.io/serviceaccount/token.

Configuration (listener)

• listener (array of objects: required) - Configuration for the listeners.

There can be one or more listener blocks at the top level. Adding a listener enables the API Proxy and enables the API proxy to use the cache, if configured. These configuration values are common to both tcp and unix listener blocks. Blocks of type tcp support the standard tcp listener options. Additionally, the role string option is available as part of the top level of the listener block, which can be configured to metrics_only to serve only metrics, or the default role, default, which serves everything (including metrics).

• type (string: required) - The type of the listener to use. Valid values are tcp and unix. Note: when using HCL this can be used as the key for the block, e.g. listener "tcp" {...}.

• address (string: required) - The address for the listener to listen to. This can either be a URL path when using tcp or a file path when using unix. For example, 127.0.0.1:8200 or /path/to/socket. Defaults to 127.0.0.1:8200.

• tls_disable (bool: false) - Specifies if TLS will be disabled.

• tls_key_file (string: optional) - Specifies the path to the private key for the certificate.

• tls_cert_file (string: optional) - Specifies the path to the certificate for TLS.

Example configuration

Here is an example of a cache configuration with the optional persist block, alongside a regular listener, and a listener that only serves metrics.

# Other Vault proxy configuration blocks
# ...

cache {
    persist = {
        type = "kubernetes"
        path = "/vault/proxy-cache/"
        keep_after_import = true
        exit_on_err = true
        service_account_token_file = "/tmp/serviceaccount/token"
    }
}

listener "tcp" {
    address = "127.0.0.1:8100"
    tls_disable = true
}

listener "tcp" {
    address = "127.0.0.1:3000"
    tls_disable = true
    role = "metrics_only"
}