Sentinel policies

Nomad Enterprise integrates with HashiCorp Sentinel for fine-grained policy enforcement. Sentinel allows operators to express their policies as code and have their policies automatically enforced. This allows operators to define a "sandbox" and restrict actions to only those compliant with policy.

The Sentinel integration builds on the ACL System. The integration provides the ability to create fine grained policy enforcements. Users must have the appropriate permissions to perform an action and are subject to any applicable Sentinel policies.

In this guide, you will create a policy and then practice applying it to a job at different enforcement levels. Finally, you'll learn more about Sentinel language specifics.

Prerequisites

The following example demonstrates how to install a Sentinel policy. It assumes that ACLs have already been bootstrapped (refer to the ACL guide), and that a NOMAD_TOKEN environment variable is set to a management token.

Create, install, and test a policy

First, create a Sentinel policy, named test.sentinel:

## Test policy always fails for demonstration purposes
main = rule { false }

Then, install this as an "advisory" policy which issues a warning on failure:

$ nomad sentinel apply -level=advisory test-policy test.sentinel
Successfully wrote "test-policy" Sentinel policy!

Use nomad job init to create a job file.

$ nomad job init
Example job file written to example.nomad.hcl

Attempt to submit that job file with nomad job run.

$ nomad job run example.nomad.hcl
Job Warnings:
1 warning(s):

* test-policy : Result: false (allowed failure based on level)

FALSE - test-policy:2:1 - Rule "main"


==> Monitoring evaluation "f43ac28d"
    Evaluation triggered by job "example"
    Evaluation within deployment: "11e01124"
    Allocation "2618f3b4" created: node "add8ce93", group "cache"
    Allocation "5c2674f2" created: node "add8ce93", group "cache"
    Allocation "9937811f" created: node "add8ce93", group "cache"
    Evaluation status changed: "pending" -> "complete"
==> Evaluation "f43ac28d" finished with status "complete"

The output indicates that the policy failed, but the job was accepted because of an "advisory" enforcement level.

Update and test the policy

Next, change test.sentinel to only allow "exec" based drivers:

# Test policy only allows exec based tasks
main = rule { all_drivers_exec }

# all_drivers_exec checks that all the drivers in use are exec
all_drivers_exec = rule {
    all job.task_groups as tg {
        all tg.tasks as task {
            task.driver is "exec"
        }
    }
}

Then updated the policy at a soft mandatory level:

$ nomad sentinel apply -level=soft-mandatory test-policy test.sentinel
Successfully wrote "test-policy" Sentinel policy!

With the new policy, attempt to submit the same job, which uses the "docker" driver:

$ nomad run example.nomad.hcl
Error submitting job: Unexpected response code: 500 (1 error(s) occurred:

* test-policy : Result: false

FALSE - test-policy:2:1 - Rule "main"
  FALSE - test-policy:6:5 - all job.task_groups as tg {
  all tg.tasks as task {
    task.driver is "exec"
  }
}

FALSE - test-policy:5:1 - Rule "all_drivers_exec"
)

The output indicates that the policy and job have failed.

Override the policy

Because the policy is failing, the job was rejected. Since the policy level is "soft-mandatory", you can override it using the -policy-override flag.

Submit the job again with the -policy-override flag set:

$ nomad job run -policy-override example.nomad.hcl
Job Warnings:
1 warning(s):

* test-policy : Result: false (allowed failure based on level)

FALSE - test-policy:2:1 - Rule "main"
  FALSE - test-policy:6:5 - all job.task_groups as tg {
  all tg.tasks as task {
    task.driver is "exec"
  }
}

FALSE - test-policy:5:1 - Rule "all_drivers_exec"


==> Monitoring evaluation "16195b50"
    Evaluation triggered by job "example"
    Evaluation within deployment: "11e01124"
    Evaluation status changed: "pending" -> "complete"
==> Evaluation "16195b50" finished with status "complete"

This time, the job was accepted but with a warning that our policy is failing but was overridden.

Extend your knowledge: policy specification

Sentinel policies are specified in the Sentinel Language. The language is designed to be understandable for people who are reading and writing policies, while remaining fast to evaluate. There is no limitation on how complex policies can be, but they are in the execution path so care should be taken to avoid adversely impacting performance.

In each scope, there are different objects made available for introspection, such a job being submitted. Policies can inspect these objects to apply fine-grained policies.

Sentinel job objects

The job object is made available to policies in the submit-job scope automatically, without an explicit import. The object maps to the JSON job specification, but fields differ slightly for better readability.

Sentinel convention for identifiers is lower case and separated by underscores. All fields on the job are accessed by the same name, converted to lower case and separating camel case to underscores. Here are some examples:

Learn more about Sentinel

For specific details about working with Sentinel, consult the nomad sentinel sub-commands and HTTP API documentation.