HashiCorp Developer AI Private beta now available Sign up today
Consul
Service Mesh Security

Control service requests with application-aware intentions

  • 24min
  • |
  • Consul
  • Interactive

Workflow

In the previous tutorial you learned how to define intentions for your services in Consul service mesh.

Intentions presented in the previous tutorial allow broad all-or-nothing access control between pairs of services and provide a secure approach for TCP-based services. For this reason they are also referred to as L4 intentions, because they operate at the transport layer of the ISO/OSI model, and are only capable of allowing or denying access to a specific service.

Most modern scenarios, however, contain HTTP-based services, such as APIs or web services, that expose multiple endpoints and that might have different access profiles depending on the source of the request or on some metadata of the request itself. This includes HTTP headers, cookies, and URL paths.

To secure such services, a simple all-or-nothing access control is often insufficient.

Consul allows to specify a different, more powerful, kind of intentions that takes into account request metadata to define request permissions. They are usually referred as L7 intentions, or application-aware intentions, because they operate at the application layer of he ISO/OSI model, and permit a fine grained filtering of the different requests.

In this tutorial, you will learn how to apply application-aware intentions to your Consul datacenter to fine tune allowed requests for an existing service.

Tutorial scenario

This tutorial uses HashiCups, a demo coffee shop application made up of several microservices running on VMs.

At the beginning of the tutorial, you have a fully deployed Consul service mesh, with Envoy sidecar proxies running alongside each service, and an API Gateway configured to permit access to the NGINX service.

The Consul service mesh is initially configured with a deny-all default policy for ACL and intentions, that disables undesired service-to-service communications, and with a set of minimal intentions that permit the HashiCups services to communicate properly.

Architecture Initial scenario Architecture Initial scenario

During the tutorial, you will verify that the API service exposes multiple endpoints, /, /api, and /health, and that the L4 intentions are not suited to selectively block requests on the different endpoints.

By the end of the tutorial, you will have learned how to apply L7 intentions to your Consul service mesh to remove access to the / and /health endpoints while allowing traffic over the /api endpoint needed for the HashiCups application.

Architecture Final scenario Architecture Final scenario

Prerequisites

This tutorial assumes you are already familiar with Consul service mesh and its core functionalities. If you are new to Consul refer to refer to the Consul Getting Started tutorials collection.

This tutorial uses an interactive lab to guide you through the process of setting up a service mesh on your VM workloads. The lab environment includes all required binaries and sample configurations.

Launch Terminal

This tutorial includes a free interactive command-line lab that lets you follow along on actual cloud infrastructure.

Configure CLI to interact with Consul

Configure your bastion host to communicate with your Consul environment using the two dynamically generated environment variable files.

$ source assets/scenario/env-consul.env && \
 source assets/scenario/env-scenario.env

After loading the needed variables, verify you can connect to your Consul datacenter.

$ consul members
Node                Address          Status  Type    Build   Protocol  DC   Partition  Segment
consul-server-0     10.0.4.207:8301  alive   server  1.16.3  2         dc1  default    <all>
gateway-api         10.0.4.27:8301   alive   client  1.16.3  2         dc1  default    <default>
hashicups-api       10.0.4.83:8301   alive   client  1.16.3  2         dc1  default    <default>
hashicups-db        10.0.4.65:8301   alive   client  1.16.3  2         dc1  default    <default>
hashicups-frontend  10.0.4.33:8301   alive   client  1.16.3  2         dc1  default    <default>
hashicups-nginx     10.0.4.130:8301  alive   client  1.16.3  2         dc1  default    <default>

Setup production intentions

Use the provided script to generate service intentions.

First, setup the output folder where to store the configuration files.

$ export OUTPUT_FOLDER=assets/scenario/conf/

Then, use the script to generate intentions.

$ bash ops/scenarios/99_supporting_scripts/generate_consul_service_intentions.sh
Parameter Check
Create global proxy configuration
Create intention configuration files

Apply the intentions to the Consul datacenter.

$ consul config write ${OUTPUT_FOLDER}global/intention-db.hcl && \
    consul config write ${OUTPUT_FOLDER}global/intention-api.hcl && \
    consul config write ${OUTPUT_FOLDER}global/intention-frontend.hcl && \
    consul config write ${OUTPUT_FOLDER}global/intention-nginx.hcl && \
    consul config delete -kind service-intentions -name "*"

This will return the following output.

Config entry written: service-intentions/hashicups-db
Config entry written: service-intentions/hashicups-api
Config entry written: service-intentions/hashicups-frontend
Config entry written: service-intentions/hashicups-nginx
Config entry deleted: service-intentions/*

Check intentions for a Consul datacenter

After setting up the prerequisites, check the intentions configuration for your Consul datacenter.

Use the consul intention command to check existing intentions.

$ consul intention list
ID  Source           Action  Destination         Precedence
    gateway-api      allow   hashicups-nginx     9
    hashicups-api    allow   hashicups-db        9
    hashicups-nginx  allow   hashicups-api       9
    hashicups-nginx  allow   hashicups-frontend  9

Create ACL token for intentions management

To configure intentions for a service, Consul includes the intentions scope for the ACL rules of a service. In this tutorial you will configure intentions for the hashicups-api service, therefore the policy will need the intentions = "write" for this service.

First define the output folder where to store the files generated.

$ export OUTPUT_FOLDER=assets/scenario/conf/

Then, create a policy file for the ACL rules you want to grant.

$ tee ${OUTPUT_FOLDER}/acl-policy-intentions-hashicups-api-l7.hcl > /dev/null << EOF
# ----------------------------------------+
# acl-token-intentions-hashicups-api-l7   |
# ----------------------------------------+

service_prefix "hashicups-api" {
  policy = "write"
  intentions = "write"
}
EOF

Tip

Consul grants permissions for creating and managing intentions based on the destination, not the source. When ACLs are enabled, services and operators must present a token linked to a policy that grants the necessary permissions to the destination service.

Create the Consul policy.

$ consul acl policy create \
    -name "hashicups-api-l7-intentions-policy" \
    -rules @${OUTPUT_FOLDER}/acl-policy-intentions-hashicups-api-l7.hcl

The output should look like the following.

ID:           75ae537c-9445-6b5d-a1ae-f3b60195f0c9
Name:         hashicups-api-l7-intentions-policy
Description:
Datacenters:
Rules:
# ----------------------------------------+
# acl-token-intentions-hashicups-api-l7   |
# ----------------------------------------+

service_prefix "hashicups-api" {
  policy = "write"
  intentions = "write"
}

Then, create a token associated with the policy.

$ consul acl token create \
    -description 'HashiCups API L7 Intentions Management token' \
    -policy-name hashicups-api-l7-intentions-policy --format json > ${OUTPUT_FOLDER}secrets/acl-token-intentions-hashicups-api-l7.json

To continue with the tutorial, using minimal permissions, export the first token as an environment variable. 

$ export CONSUL_HTTP_TOKEN=`cat ${OUTPUT_FOLDER}secrets/acl-token-intentions-hashicups-api-l7.json | jq -r ".SecretID"`

Explore HashiCups API service endpoints

The HashiCups API service is configured to be reachable only from the NGINX node in this scenario. This implementation allows all-or-nothing access control between pairs of services and is a secure approach for TCP-based services.

HTTP-based services, like the HashiCups API service, often offer a more complex range of requests and can expose multiple endpoints to implement different request types or to expose different content.

You will now explore the different endpoints provided by the HashiCups API service.

Explore the HashiCups API /api path

Click on the NGINX tab.

Verify that the API service is exposed locally on the NGINX node on the port specified in the service upstream definition, 8081.

$ netstat -natp | grep LISTEN

If the service mesh is configured correctly you should get an output similar to the following, with one entry, on port 8081 corresponding to the API service.

tcp        0      0 0.0.0.0:21000           0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:8600          0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:8502          0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:8500          0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:19000         0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:3000          0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.11:35427        0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:12346         0.0.0.0:*               LISTEN      160/grafana-agent
tcp        0      0 127.0.0.1:12345         0.0.0.0:*               LISTEN      160/grafana-agent
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      -
tcp        0      0 192.168.32.11:8301      0.0.0.0:*               LISTEN      -
tcp        0      0 127.0.0.1:8081          0.0.0.0:*               LISTEN      -
tcp6       0      0 :::22                   :::*                    LISTEN      -

Run a query to the HashiCups API service from the NGINX node.

$ curl --silent 'http://localhost:8081/api' \
      -H 'Accept-Encoding: gzip, deflate, br' \
      -H 'Content-Type: application/json' \
      -H 'Accept: application/json' \
      -H 'Connection: keep-alive' \
      -H 'DNT: 1' \
      -H 'Origin: http://localhost:8081' \
      --data-binary '{"query":"mutation{ pay(details:{ name: \"nic\", type: \"mastercard\", number: \"1234123-0123123\", expiry:\"10/02\",    cv2: 1231, amount: 12.23 }){id, card_plaintext, card_ciphertext, message } }"}' --compressed | jq

A successful request will return an output similar to the following:

{
  "data": {
    "pay": {
      "id": "b7912087-80c8-4520-99bb-79a89f1b611e",
      "card_plaintext": "1234123-0123123",
      "card_ciphertext": "Encryption Disabled",
      "message": "Payment processed successfully, card details returned for demo purposes, not for production"
    }
  }
}

In the previous command, you used the /api path for the request, but the API service also exposes other paths.

Explore the HashiCups API root path

The default path, /, provides a number of information about the API service configuration.

$ curl --silent 'http://localhost:8081/'

Example output.

<!DOCTYPE html>
<html>
<head>
    <meta charset=utf-8/>
    <meta name="viewport" content="user-scalable=no, initial-scale=1.0, minimum-scale=1.0, maximum-scale=1.0, minimal-ui">
    <link rel="shortcut icon" href="https://graphcool-playground.netlify.com/favicon.png">
    <link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/graphql-playground-react@1.7.20/build/static/css/index.css"
        integrity="sha256-cS9Vc2OBt9eUf4sykRWukeFYaInL29&#43;myBmFDSa7F/U=" crossorigin="anonymous"/>
    <link rel="shortcut icon" href="https://cdn.jsdelivr.net/npm/graphql-playground-react@1.7.20/build/favicon.png"
        integrity="sha256-GhTyE&#43;McTU79R4&#43;pRO6ih&#43;4TfsTOrpPwD8ReKFzb3PM=" crossorigin="anonymous"/>
    <script src="https://cdn.jsdelivr.net/npm/graphql-playground-react@1.7.20/build/static/js/middleware.js"
        integrity="sha256-4QG1Uza2GgGdlBL3RCBCGtGeZB6bDbsw8OltCMGeJsA=" crossorigin="anonymous"></script>
    <title>Playground</title>
</head>
<body>
<style type="text/css">
    html { font-family: "Open Sans", sans-serif; overflow: hidden; }
    body { margin: 0; background: #172a3a; }
</style>
<div id="root"/>
<script type="text/javascript">
    window.addEventListener('load', function (event) {
        const root = document.getElementById('root');
        root.classList.add('playgroundIn');
        const wsProto = location.protocol == 'https:' ? 'wss:' : 'ws:'
        GraphQLPlayground.init(root, {
            endpoint: location.protocol + '//' + location.host + '\/api',
            subscriptionsEndpoint: wsProto + '//' + location.host + '\/api',
           shareEnabled: true,
            settings: {
                'request.credentials': 'same-origin'
            }
        })
    })
</script>
</body>
</html>

Explore the HashiCups API /health path

The API service also exposes a simple /health endpoint that provides a summary of the state of the service.

$ curl --silent 'http://localhost:8081/health'

If the service is up and running the call will respond with a simple message.

ok

In some cases it can respond with a message informing about the temporary state of one of its sub-components.

error. the following services are down: product-api

Security considerations

When allowing connections towards an HTTP-based service, you also allow access to all endpoints exposed by that service. While some of these endpoints might be necessary for the correct behavior of the service, or might be needed by the service's downstreams, some of them might be unnecessary or might also expose sensitive information to external users.

In this case, allowing general connections towards the HashiCups API service allows access to all paths, including the / and /health paths, which are not required for the correct behavior of HashiCups. Extraneous application paths such as these could represent a security risk.

Architecture Initial scenario - Explicit Architecture Initial scenario - Explicit

For cases like this, where a simple yes/no configuration is not enough to guarantee security, Consul allows to specify a different, more powerful, kind of intentions.

They are usually referred as L7 intentions, or application-aware intentions, because they operate at the application layer of he ISO/OSI model and permit a fine grained filtering of the different requests.

In the rest of the tutorial, you will learn how to apply application-aware intentions to your Consul datacenter to fine tune allowed requests for the API service.

Define and apply L7 intentions

The use of L7 intentions is only available for destination services using an HTTP-based protocol.

Click on the Bastion Host tab.

In this case the destination service is hashicups-api and you can verify the configuration for it using the consul config command.

$ consul config read -kind service-defaults -name hashicups-api
Error reading config entry service-defaults/hashicups-api: Unexpected response code: 404 (Config entry not found for "service-defaults" / "hashicups-api")

The 404 message indicates that there is no configuration present for the hashicups-api service. The service can be regarded as a TCP-based service.

Tip

Service protocol can also be specified using proxy-defaults and, in that case, they will apply to all services in the service mesh. The protocol specified for individual service instances in the service-defaults configuration entry takes precedence over the globally-configured value set in the proxy-defaults.

Define service protocol for hashicups-api

To apply L7 intentions to hashicups-api you will have to define it as an HTTP-based service first.

Define the output folder for the configuration file to be created.

$ export OUTPUT_FOLDER=assets/scenario/conf/

Then, create the configuration file that defines the Protocol value for the service.

$ tee ${OUTPUT_FOLDER}config-global-default-hashicups-api.hcl > /dev/null << EOF
Kind      = "service-defaults"
Name      = "hashicups-api"
Protocol  = "http"
EOF

Finally, apply the configuration to your Consul datacenter.

$ consul config write ${OUTPUT_FOLDER}config-global-default-hashicups-api.hcl
Config entry written: service-defaults/hashicups-api | jq

Once the configuration is applied, verify the hashicups-api service now gets a configuration applied that defines it as an HTTP-based service.

$ consul config read -kind service-defaults -name hashicups-api

The command will return the service-defaults configuration applied to the hashicups-api service. Notice that the output is not empty this time and that the Protocol value is set to http.

{
  "Kind": "service-defaults",
  "Name": "hashicups-api",
  "Protocol": "http",
  "TransparentProxy": {},
  "MeshGateway": {},
  "Expose": {},
  "CreateIndex": 1039,
  "ModifyIndex": 1039
}

Define L7 intentions for hashicups-api

Once the API service is configured to be HTTP-based, it is possible to apply the L7 intention to it.

First, create the configuration file to describe the intention.

$ tee ${OUTPUT_FOLDER}config-intention-hashicups-api.hcl > /dev/null << EOF
Kind = "service-intentions"
Name = "hashicups-api"
Sources = [
  {
    Name = "hashicups-nginx"
    Permissions = [
      {
        Action = "allow"
        HTTP {
          PathExact = "/api"
        }
      }
    ]
  },
  {
    Name = "*"
    Permissions = [
      {
        Action = "deny"
        HTTP {
          PathExact = "/health"
          Methods   = ["GET"]
        }
      },
      {
        Action = "deny"
        HTTP {
          PathExact = "/"
          Methods   = ["GET"]
        }
      }
    ]
  },
  # NOTE: a default catch-all based on the default ACL policy will apply to
  # unmatched connections and requests. Typically this will be DENY.
]
EOF

Then, apply the configuration using the consul config command.

$ consul config write ${OUTPUT_FOLDER}config-intention-hashicups-api.hcl 
Config entry written: service-intentions/hashicups-api

Note

Notice that the intention definition file also defines the intention between hashicups-nginx and hashicups-api even if an intention already exists for that connection in Consul. This is because when writing a configuration into Consul, the Name parameter is used to describe the configuration and, in case an existing configuration with that name is already in place, it will get overwritten by the new one.

Check intentions in your Consul datacenter

After applying the desired intention, check the new intentions configuration for your Consul datacenter.

Use the consul intention command to check existing intentions.

$ consul intention list
ID  Source           Action  Destination    Precedence
    hashicups-api    allow   hashicups-db   9
    hashicups-nginx          hashicups-api  9
    *                        hashicups-api  8

Note

The intentions shown by the command depend on the ACL token present in the CONSUL_HTTP_TOKEN variable. If using a limited token, that only gives permissions over the hashicups-api service, only the intentions relative to that service will be returned by the command.

To get more details over the specific intentions in place for hashicups-api, use the consul config command.

$ consul config read -kind service-intentions -name hashicups-api

This will provide you with the full details of all service intentions regarding the hashicups-api service.

{
    "Kind": "service-intentions",
    "Name": "hashicups-api",
    "Sources": [
        {
            "Name": "hashicups-nginx",
            "Permissions": [
                {
                    "Action": "allow",
                    "HTTP": {
                        "PathExact": "/api"
                    }
                }
            ],
            "Precedence": 9,
            "Type": "consul"
        },
        {
            "Name": "*",
            "Permissions": [
                {
                    "Action": "deny",
                    "HTTP": {
                        "PathExact": "/health",
                        "Methods": [
                            "GET"
                        ]
                    }
                },
                {
                    "Action": "deny",
                    "HTTP": {
                        "PathExact": "/",
                        "Methods": [
                            "GET"
                        ]
                    }
                }
            ],
            "Precedence": 8,
            "Type": "consul"
        }
    ],
    "CreateIndex": 2293,
    "ModifyIndex": 2976
}

Verify connections are now secured

Now that all intentions are in place, it is time to verify that the service works as expected and that the undesired endpoints are now unavailable from other nodes.

Check HashiCups UI

The first check to perform is to make sure that the HashiCups application is still functional.

Click on the HashiCups tab.

Confirm that HashiCups still works with the new intentions in place.

HashiCups UI HashiCups UI

Test service to service connection

Now, after confirming that the application is working as expected, you can verify that only the /api endpoint is reachable from the NGINX node.

Click on the NGINX tab.

Verify the connection to the API service from the NGINX node.

$ curl --silent 'http://localhost:8081/api' \
      -H 'Accept-Encoding: gzip, deflate, br' \
      -H 'Content-Type: application/json' \
      -H 'Accept: application/json' \
      -H 'Connection: keep-alive' \
      -H 'DNT: 1' \
      -H 'Origin: http://localhost:8081' \
      --data-binary '{"query":"mutation{ pay(details:{ name: \"nic\", type: \"mastercard\", number: \"1234123-0123123\", expiry:\"10/02\",    cv2: 1231, amount: 12.23 }){id, card_plaintext, card_ciphertext, message } }"}' --compressed | jq

If the intentions are being enforced properly, the command will return an output similar to the following:

{
  "data": {
    "pay": {
      "id": "b7912087-80c8-4520-99bb-79a89f1b611e",
      "card_plaintext": "1234123-0123123",
      "card_ciphertext": "Encryption Disabled",
      "message": "Payment processed successfully, card details returned for demo purposes, not for production"
    }
  }
}

Next, you can test if the / endpoint is reachable.

$ curl --silent 'http://localhost:8081/'

If the intentions are being enforced properly, the command will return an output similar to the following:

RBAC: access denied

This means that the request is being blocked by the Role Based Access Control (RBAC) enforced by Consul.

Finally, you can test if the /health endpoint is reachable.

$ curl --silent 'http://localhost:8081/health'

If the intentions are being enforced properly, the command will return an output similar to the following:

RBAC: access denied

This means that the request is being blocked by the Role Based Access Control (RBAC) enforced by Consul.

Next steps

In this tutorial you learned how to use L7 intentions in Consul service mesh to control service requests at the application level.

More specifically, you learned to:

  • Verify the existing intentions in a Consul datacenter
  • Modify the service default protocol from TCP to HTTP
  • Create an L7 intention definition
  • Apply the intention to Consul datacenter using API endpoints or Consul CLI commands

For more information about the topics covered in this tutorial, refer to the following resources:

Previous

Traffic communication with L4 intentions

 Next

Explore tutorial library

This tutorial also appears in:

  • 4 tutorials
    Secure services
    Secure service to service traffic in Consul service mesh using service intentions, both at L4 and L7 level, and Access Control Lists (ACLs) to avoid unidentified traffic. Include new services in Consul service mesh gradually using permissive mTLS.
    • Consul

Workflow