Database secrets engine with MongoDB

22min
|
Vault

Deployment

Data protection is a top priority, and database credential rotation is a critical part of any data protection initiative. When hackers attack a system, continuous credential rotation becomes necessary, and you should strive to automate the process.

Vault's database secrets engine generates database credentials dynamically based on user-defined roles. The database administrator can pre-define the time-to-live (TTL) of the database credentials to enforce its validity so that they are automatically revoked when they expire.

Each app instance can get unique credentials that they don't have to share. By making those credentials short-lived, you reduce the chance of compromise. If an attacker compromises an app, the credentials used by the app can be revoked rather than changing more global sets of credentials.

Personas

The end-to-end scenario described in this tutorial involves two personas:

admin with privileged permissions to configure secrets engines
Vault clients (users, apps, systems, etc.) read the secrets from Vault

Prerequisites

Note

This lab works on macOS using x86_64 based processors. If you are running macOS on an Apple silicon processor, consider using a x86_64 based Linux virtual machine in your preferred cloud provider.

To perform the tasks described in this tutorial, you need to have:

Docker to run MongDB
Vault installed
jq installed
ngrok installed and configured with an auth token (required for HCP Vault Dedicated)

Lab setup

Start MongoDB

The tutorial requires a MongoDB database. Docker provides a MongoDB server image that you will use in this lab.

Note

This tutorial also works for an existing MongoDB database given appropriate credentials and connection information.

Create a MongoDB database with a root user named mdbadmin with the password hQ97T9JJKZoqnFn2NXE.

$ docker run -d \
    -p 0.0.0.0:27017:27017 -p 0.0.0.0:28017:28017 \
    --name=mongodb \
    -e MONGO_INITDB_ROOT_USERNAME="mdbadmin" \
    -e MONGO_INITDB_ROOT_PASSWORD="hQ97T9JJKZoqnFn2NXE" \
    mongo

The database is now available to experiment with the database secrets engine.

Start Vault

Note

If you do not have access to an HCP Vault Dedicated cluster, visit the Create a Vault Cluster on HCP tutorial.

Launch the HCP Portal and login.
Click Vault in the left navigation pane.
In the Vault clusters pane, click vault-cluster.
Under Cluster URLs, click Public Cluster URL.
In a terminal, set the VAULT_ADDR environment variable to the copied address.
```
$ export VAULT_ADDR=<Public_Cluster_URL>
```
Return to the Overview page and click Generate token.
Vault generates a new token.
Copy the Admin Token.
Return to the terminal and set the VAULT_TOKEN environment variable.
```
$ export VAULT_TOKEN=<token>
```
Set the VAULT_NAMESPACE environment variable to admin.
```
$ export VAULT_NAMESPACE=admin
```
The admin namespace is the top-level namespace automatically created by HCP Vault. All CLI operations default to use the namespace defined in this environment variable.

Type vault status to verify your connectivity to the Vault cluster.

$ vault status

Key                      Value
---                      -----
Recovery Seal Type       shamir
Initialized              true
Sealed                   false
Total Recovery Shares    1
Threshold                1
Version                  1.9.2+ent
Storage Type             raft
...snipped...

You must establish a tunnel for Vault Dedicated to interact with resources running on your local machine.

In another terminal, start ngrok and connect to postgreSQL.

$ ngrok tcp 127.0.0.1:27017

Example output:

ngrok                                                                                                                                                              (Ctrl+C to quit)

Session Status                online
Account                       username (Plan: Free)
Update                        update available (version 3.0.5, Ctrl-U to update)
Version                       3.0.3
Region                        United States (us)
Latency                       32.791235ms
Web Interface                 http://127.0.0.1:4040
Forwarding                    tcp://d12b-34-567-89-10.ngrok.io:12345 -> 127.0.0.1:28017

Connections                   ttl     opn     rt1     rt5     p50     p90
                              0       0       0.00    0.00    0.00    0.00

Copy the ngrok forwarding address.
Return to the terminal where you set the VAULT_ADDR environment variable and set an environment variable for the ngrok address. Do not include tcp://.
```
$ export MONGODB_URL=<actual-address-from-ngrok>
```

You are ready to proceed with the tutorial.

Scenario introduction

In this tutorial, you will configure the MongoDB secrets engine and create a "tester" role that has read and write permissions.

Enable the database secrets engine - admin task
Configure MongoDB secrets engine - admin task
Create a role - admin task
Request MongoDB credentials - Vault clients to perform

This tutorial is running Vault in development mode, and use the root token. Be sure to read the Policy requirements section at the end.

Enable the database secrets engine

(Persona: admin)

The database secrets engine generates database credentials dynamically based on configured roles.

Open a web browser and launch the Vault UI (the address is the same as VAULT_ADDR).
Enter the token in the Token field and click Sign In (the token is the same as VAULT_TOKEN).
Select Enable new engine.
Select Databases from the list, and then click Next.
Enter mongodb in the Path field.
Click Enable Engine to complete.

Enable the database secrets engine at the mongodb/ path.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --header "X-Vault-Namespace: $VAULT_NAMESPACE" \
    --request POST \
    --data '{"type":"database"}' \
    $VAULT_ADDR/v1/sys/mounts/mongodb

You enabled the database secrets engine at mongodb/.

Configure MongoDB secrets engine

(Persona: admin)

The database secrets engine supports popular databases through a plugin interface. To use a MongoDB database with the secrets engine requires further configuration with the mongodb-database-plugin plugin, and connection information.

Select Create connection in the Connections tab for mongodb.
Select MongoDB from the Database plugin drop-down list.
Enter mongo-test in the Connection Name field.
Enter the following in the Connection URL field replacing <host:port> with the value from the MONGODB_URL environment variable.
```
mongodb://{{username}}:{{password}}@<host:port>/admin?tls=false
```
Enter mdbadmin in the Username and hQ97T9JJKZoqnFn2NXE in the Password text fields.
Click Create database.
When prompted, click Enable without rotating to continue.
This is an optional step; you should rotate the root user's password in production use. The rest of this tutorial relies on the password you set when staring the MongoDB container, so you should skip this step to complete the tutorial.
Read the Database Root Credential Rotation tutorial to learn about rotating the root credential after the initial configuration of each database.
Select mongodb.
Select the Overview tab. Once you set up a database connection, the overview page displays the current configuration summary.

Configure the database secrets engine with the connection credentials for the MongoDB database.

$ vault write mongodb/config/mongo-test \
      plugin_name=mongodb-database-plugin \
      allowed_roles="tester" \
      connection_url="mongodb://{{username}}:{{password}}@$MONGODB_URL/admin?tls=false" \
      username="mdbadmin" \
      password="hQ97T9JJKZoqnFn2NXE"

Read the Database Root Credential Rotation tutorial to learn about rotating the root credential after the initial configuration of each database.

Create an API request payload with the database configuration.

$ tee payload.json <<EOF
{
  "plugin_name": "mongodb-database-plugin",
  "allowed_roles": "tester",
  "connection_url": "mongodb://{{username}}:{{password}}@$MONGODB_URL/admin?tls=false",
  "username": "mdbadmin",
  "password": "hQ97T9JJKZoqnFn2NXE"
}
EOF

Configure the database secrets engine with the connection credentials for the MongoDB database.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --header "X-Vault-Namespace: $VAULT_NAMESPACE" \
    --request POST \
    --data @payload.json \
    $VAULT_ADDR/v1/mongodb/config/mongo-test

Read the Database Root Credential Rotation tutorial to learn about rotating the root credential after the initial configuration of each database.

The Vault generated password has a default pattern which may not adhere to your organization's password requirements. Read the User Configurable Password Generation for Secret Engines tutorial to learn about configuring the Vault-generated database password schema.

Create a role

(Persona: admin)

A role is a logical name within Vault that maps to database credentials. Some applications may just need read permissions, and others may require read and write permissions. Practice the principle of least privilege and create a role appropriately for each database client.

Note

Important: when you define the role in a production deployment, you must create user creation_statements and revocation_statements, which are valid for the database you've configured. If you do not specify statements appropriate to creating, revoking, or rotating users, Vault inserts generic statements which can be unsuitable for your deployment.

In this step, you will create a "tester" role with time-to-live (TTL) set to 1 hour, and the maximum TTL is 24 hours. This allows Vault to revoke the credentials automatically once they reach the TTL.

In the mongodb overview page, select Create new.
Enter tester in the Role name field.
Enter mongo-test in the Database name field.
Select dynamic from the Type of role drop-down list.
Verify Generated credential's Time-to-Live (TTL) and Generated credential's maximum Time-to-Live (Max TTL) is to set the value to 1 hour and 1 day respectively.

Enter the following in the Creation statement field.

{
  "db": "admin",
  "roles": [
    {
      "role": "readWrite"
    },
    {
      "role": "read",
      "db": "foo"
    }
  ]
}

The Create Role page should look as below: Create a Role

Click Create Role to complete.

Create the role named tester.

$ vault write mongodb/roles/tester \
    db_name=mongo-test \
    creation_statements='{ "db": "admin", "roles": [{ "role": "readWrite" }, {"role": "read", "db": "foo"}] }' \
    default_ttl="1h" \
    max_ttl="24h"

Verify that the tester role exists.

$ vault list mongodb/roles

Keys
----
tester

Create an API request payload containing the database role definition.

$ tee payload-role.json <<EOF
{
    "db_name": "mongo-test",
    "creation_statements": [
    "{ \"db\": \"admin\", \"roles\": [{ \"role\": \"readWrite\" }, {\"role\": \"read\", \"db\": \"foo\"}] }"
    ],
    "default_ttl": "1h",
    "max_ttl": "24h"
}
EOF

Create the role named tester that creates credentials with the creation_statements defined in the payload.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --header "X-Vault-Namespace: $VAULT_NAMESPACE" \
    --request POST --data @payload-role.json \
    $VAULT_ADDR/v1/mongodb/roles/tester

Vault can now dynamically generate database credentials useful for connecting to the mongodb database.

Request MongoDB credentials

(Persona: Vault clients)

To connect to the MongoDB, Vault clients request Vault to dynamically generate the database credentials based on its role. In this case, the tester role.

This step uses the root token for demonstration. In reality, Vault clients would never use Vault's root token. After learning the basic steps, continue onto the Policy requirements section.

Return to the mongo-test configuration page by selecting mongodb > mongo-test.
Select the tester role.
Select Generate credentials.
This presents the generated lease. Click on the copy to clipboard icon to copy the generated username and password.

NOTE: From the mongodb overview page, enter the role name in the Get Credentials field and click Get Credentials to do the same. The Get Credentials works for any role configured on the mongodb.

Read credentials from the tester database role.

$ vault read mongodb/creds/tester

Example output:

Key                Value
---                -----
lease_id           mongodb/creds/tester/fyF5xDomnKeCHNZNQgStwBKD
lease_duration     1h
lease_renewable    true
password           A1a-ckirtymYaXACpIHn
username           v-token-tester-6iRIcGv8tLpu816oblPY-1556567086

Read credentials from the tester database role.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --header "X-Vault-Namespace: $VAULT_NAMESPACE" \
    --request GET \
    $VAULT_ADDR/v1/mongodb/creds/tester | jq .data

Note

This example uses jq to process the JSON output for readability.

Example output:

{
  "password": "tHjmYnh7vb7zIXkuE-UU",
  "username": "v-token-tester-VX26utiXIlrseoBbgZuz-1615452504"
}

The MongoDB credentials are a username and a password. Vault identifies the credentials by their lease ID.

Read the customize the generated username schema section for additional discussion.

Validation

Connect to the MongoDB database using the Vault generated credentials.

$ docker exec -it mongodb mongosh --username <username> --password <password>

Example:

$ docker exec -it mongodb mongosh \
    --username v-token-tester-fO7smPyTgCrySIgQNrst-1615451034 \
    --password IePqQ26-EyIk6zEA637w

MongoDB shell version v4.4.4
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("aabf31e2-27f9-4ac5-9145-710dbd84ef8d") }
MongoDB server version: 4.4.4

You can list databases.

> show dbs;
admin  0.000GB

Enter exit to exit out of the Docker container.

Policy requirements

Each persona requires a different set of capabilities, and you use ACL policies to enforce those capabilities. If you are not familiar with policies, complete the policies tutorial.

Example policy for admin

The admin persona needs to be able to set up the database secrets engine.

Enable the database secrets engine at mongodb (sys/mounts/mongodb)
Configure the MongoDB connection (mongodb/config/mongo-test)
Create a role (mongodb/roles/tester)

An admin needs to set up the Vault clients so they can communicate with Vault.

Create policies for the Vault clients (sys/policies/acl/<policy_name>)
Enable an auth method for the Vault clients to login with Vault (sys/auth/<auth_method>)
Configure the auth method (auth/<auth_method>)

# Enable secrets engines at any path
path "sys/mounts/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Manage the database secrets engine enabled at `mongodb` path
path "mongodb/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

# Create ACL policies for Vault clients
path "sys/policies/acl/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

#-----------------------------------------
# Example: Use AppRole auth method
#-----------------------------------------
# Enable approle auth method at 'approle'
path "sys/auth/approle" {
  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

# List all enabled auth methods
path "sys/auth" {
  capabilities = [ "read", "list" ]
}

# Create and manage roles
path "auth/approle/role/*" {
  capabilities = [ "create", "read", "update", "delete", "list" ]
}

#------------------------------------------
# Create token for testing
#------------------------------------------
# Create tokens
path "auth/token/create" {
  capabilities = [ "create", "read", "update", "delete", "list", "sudo" ]
}

You should be able to complete this tutorial using a token with the above policy attached. If you are not familiar with creating policies, read the Vault Policies tutorial.

Example policy for Vault clients

The Vault clients persona needs to be able to request credentials from Vault.

# Required: Get credentials from the database secrets engine for 'tester' role.
path "mongodb/creds/tester" {
  capabilities = [ "read", "update"]
}

# Recommended: List all dynamic and static roles
path "mongodb/roles" {
  capabilities = [ "list" ]
}

path "mongodb/static-roles" {
  capabilities = [ "list" ]
}

Create a client policy

Select Policies and then Create ACL policy.
Enter clients in the Name field.

Copy the policy and paste it into the Policy text box.

# Required: Get credentials from the database secrets engine for 'tester' role.
path "mongodb/creds/tester" {
  capabilities = [ "read", "update"]
}

# Recommended: List all dynamic and static roles
path "mongodb/roles" {
  capabilities = [ "list" ]
}

path "mongodb/static-roles" {
  capabilities = [ "list" ]
}

Click Create policy.
Log out of the Vault UI and switch to the terminal session you started in the Lab setup section.

Create a token with the clients policy attached.

$ vault token create -policy=clients -ttl=8h

Example output:

Key                  Value
---                  -----
token                hvs.ad3DG8zxyPcqq4cvrMzsIxr9
token_accessor       lUcfmpppZVO1Sr6f6gwXznaF
token_duration       8h
token_renewable      true
token_policies       ["clients" "default"]
identity_policies    []
policies             ["clients" "default"]

Copy the generated token value. In this example, the token is hvs.ad3DG8zxyPcqq4cvrMzsIxr9.

Return to the Vault UI and enter the generated token to sign in.
Select mongodb/.
Under Get Credentials, enter tester in the Role to use field.
Click Get Credentials to display the credentials.

Create an clients policy.

$ vault policy write clients -<<EOF
# Required: Get credentials from the database secrets engine for 'tester' role.
path "mongodb/creds/tester" {
  capabilities = [ "read", "update"]
}

# Recommended: List all dynamic and static roles
path "mongodb/roles" {
  capabilities = [ "list" ]
}

path "mongodb/static-roles" {
  capabilities = [ "list" ]
}
EOF

Create a token with the clients policy attached and save it as an environment variable.

$ CLIENT_TOKEN=$(vault token create -policy=clients -ttl=8h -format=json | jq -r '.auth | .client_token')

Verify the CLIENT_TOKEN has the clients policy attached.

$ vault token lookup $CLIENT_TOKEN

Key                 Value
---                 -----
accessor            XqYk8PShzsMYvLtJY7xZgp3I.Xvmlx
creation_time       1658500884
creation_ttl        8h
display_name        token
entity_id           bfa78daa-8b65-1f5f-1167-7768b68a68bc
expire_time         2022-07-22T22:41:24.092104544Z
explicit_max_ttl    0s
id                  hvs.CAESIC4h7k8Yc7hawkem_oEDBmJg7pNv3jWgc5qRq_FoDJ1oGicKImh2cy5Ja1hKMEdpTXEyVnc4OWkyWTNCOHpmS0QuWHZtbHgQ1QM
issue_time          2022-07-22T14:41:24.092141504Z
meta                <nil>
namespace_path      admin/
num_uses            0
orphan              false
path                auth/token/create
policies            [clients default]
renewable           true
ttl                 7h46m52s
type                service

Review the policies value to find the clients and default policies attached to the token.

Use the CLIENT_TOKEN to request new credentials from the tester role.

$ VAULT_TOKEN=$CLIENT_TOKEN vault read mongodb/creds/tester

Vault outputs the credentials.

Example output:

Key                Value
---                -----
lease_id           mongodb/creds/tester/Ktwvj5K0iIE05LDlGjl0391A.3EPkJ
lease_duration     1h
lease_renewable    true
password           Q-E5Xggr-40mrfz3xHn5
username           v-token-tester-IgB57Tnu3G3QylDsuXso-1658773433

Create an API request payload containing the client policy.

$ tee payload-policy.json <<EOF
{"policy": "# Required: Get credentials from the database secrets engine for 'tester' role.\npath \"mongodb/creds/tester\" {\n  capabilities = [ \"read\", \"update\"]\n}\n\n# Recommended: List all dynamic and static roles\npath \"mongodb/roles\" {\n  capabilities = [ \"list\" ]\n}\n\npath \"mongodb/static-roles\" {\n  capabilities = [ \"list\" ]\n}\n"}
EOF

Create the clients policy defined in payload-policy.json

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --header "X-Vault-Namespace: $VAULT_NAMESPACE" \
    --request POST --data @payload-policy.json \
    $VAULT_ADDR/v1/sys/policies/acl/clients

Create an API request payload.

$ tee payload-token.json <<EOF
{"policies": "clients"}
EOF

Create a token with the clients policy attached and save it as an environment variable.

$ CLIENT_TOKEN=$(curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --header "X-Vault-Namespace: $VAULT_NAMESPACE" \
    --request POST \
    --data @payload-token.json \
    $VAULT_ADDR/v1/auth/token/create | jq -r '.auth | .client_token')

Create an API payload.

$ tee payload-readtoken.json <<EOF
{"token": "$CLIENT_TOKEN"}
EOF

Verify the CLIENT_TOKEN has the clients policy attached.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --header "X-Vault-Namespace: $VAULT_NAMESPACE" \
    --request POST \
    --data @payload-readtoken.json \
    $VAULT_ADDR/v1/auth/token/lookup | jq -r '.data | .policies'

Example output:

[
  "clients",
  "default"
]

Use the CLIENT_TOKEN to request new credentials from the tester role.

curl --header "X-Vault-Token: $CLIENT_TOKEN" \
   --header "X-Vault-Namespace: $VAULT_NAMESPACE" \
   --request GET \
   $VAULT_ADDR/v1/mongodb/creds/tester | jq -r '.data'

Example output:

{
  "password": "iZGfE64P5tcZKfUeJAt-",
  "username": "v-token-tester-xFsHwJ2A8zlBGyilddef-1658773228"
}

Customize the generated username schema

In the request MongoDB credentials section, you saw the Vault generated username and password. The generated username resembles v-token-tester-6iRIcGv8tLpu816oblPY-1556567086. This may be less obvious to differentiate a username from another when you are auditing the database access.

Update the mongo-test connection configuration to specify that the generated database username to have the format of mongo-<role_name>-<random_char_length_8>.

In this tutorial, you created a tester role, so the <role_name> is tester.

Update the mongo-test connection configuration with username template.

$ vault write mongodb/config/mongo-test \
      plugin_name=mongodb-database-plugin \
      allowed_roles="tester" \
      connection_url="mongodb://{{username}}:{{password}}@$MONGODB_URL/admin?tls=false" \
      username="mdbadmin" \
      password="hQ97T9JJKZoqnFn2NXE" \
      username_template="mongo-test-{{.RoleName}}-{{random 8}}"

The username_template parameter specifies the username format ("mongo-test-{{.RoleName}}-{{random 8}}"). The {{.RoleName}} returns the role name (tester) used to request a lease. The {{random 8}} returns 8 random characters.

Request a new set of credentials.

$ vault read mongodb/creds/tester

Example output:

Key                Value
---                -----
lease_id           mongodb/creds/tester/JPQtefXthLkh7hE8vPpBY5Zw
lease_duration     24h
lease_renewable    true
password           GHVxvy6gDf7zD9Kxk-8f
username           mongo-test-tester-vLjw7VJI

The generated username (mongo-test-tester-vLjw7VJI) adheres to the username template.

Create the request payload to include a username template.

$ tee payload.json <<EOF
{
    "plugin_name": "mongodb-database-plugin",
    "allowed_roles": "tester",
    "connection_url": "mongodb://{{username}}:{{password}}@$MONGODB_URL/admin?tls=false",
    "username": "mdbadmin",
    "password": "hQ97T9JJKZoqnFn2NXE",
    "username_template": "mongo-test-{{.RoleName}}-{{random 8}}"
}
EOF

Update the mongo-test connection configuration with username template.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --header "X-Vault-Namespace: $VAULT_NAMESPACE" \
    --request POST \
    --data @payload.json \
    $VAULT_ADDR/v1/mongodb/config/mongo-test

Request a new set of credentials.

$ curl --header "X-Vault-Token: $VAULT_TOKEN" \
    --header "X-Vault-Namespace: $VAULT_NAMESPACE" \
    --request GET \
    $VAULT_ADDR/v1/mongodb/creds/tester | jq .data

Example output:

{
  "password": "wrcQNdea7jN2c6r1sVi-",
  "username": "mongo-test-tester-kJ1mfMqM"
}

The generated username (mongo-test-tester-kJ1mfMqM) adheres to the username template.

To customize the password rules, read the User Configurable Password Generation for Secret Engines tutorial.

Next steps

Resources are available to help integrate your applications with a Vault database secrets engine. You can use these resources, and your existing applications will require little to no code changes to work with Vault.

Refer to the following tutorials:

Help and reference

Couchbase secrets engine

IBM Db2 credential management