PKI secrets engine - Certificate Issuance External Policy Service (CIEPS)

This document covers high-level architecture and service APIs used by the Vault PKI Secrets Engine when communicating with the Certificate Issuance External Policy Service (CIEPS) Enterprise.

What is Certificate Issuance External Policy Service (CIEPS)?

Hashicorp Vault's PKI Secrets Engine has a mechanism for issuing leaf certificates with arbitrary structure: /pki/sign-verbatim. This requires an organization to run an application/user-accessible service for authenticating, authorizing, and validating certificate issuance requests (potentially handling key pair generation as well), before asking PKI to sign the resulting CSR and leaf certificate with its own highly-privileged Vault token. If any attribute is missing from the original requester's CSR, the original service must reject the request as sign-verbatim does not give the controlling service the ability to modify the request.

The Certificate Issuance External Policy Service (CIEPS) Enterprise protocol solves this by placing the validation and certificate templating gated behind the PKI, solving:

Auditing, so the original requester is still identified and both the original request and subsequent response are tracked.
Central access, so applications only need to use a new URL for requesting certificates.
Certificate modification, so customization of the requester's submission can be exposed to this external service.
External validation, when compared to the Role-based system, as the CIEPS implementation can reach out to customer-defined external systems for validation.

Either of these two mechanisms allow an organization to leverage the Vault PKI Secrets Engine to build their own flexible issuance control architecture, leveraging Vault as a PKI-as-a-Service platform. However, CIEPS grants far greater control to the organization than the sign-verbatim approach.

Custom policy with `sign-verbatim`

With sign-verbatim, the policy validation service must sit in front of Vault, processing requests from the user (which cannot use Vault authentication and needs to authenticate themselves separately to this service). This RA service then handles its own authentication to Vault, which provides the signing capabilities via the PKI plugin.

When the application retains control over its own key material by providing a CSR, the policy service cannot modify the requested CSR and thus cannot modify the resulting certificate. It can only approve or deny requests without allowing operators to hide implementation details from calling applications. This is because PKI's sign-verbatim endpoint lacks the ability for the Vault API caller (in this case, the fronting policy service) to modify the certificate independent of the provided CSR.

If, however, the policy service can control key material (and this is an acceptable risk to the organization), the policy service could modify requests on behalf of the calling application. However, this still requires the external application to know how to authenticate to this external policy service.

Additionally, to ensure compatibility with Vault, this policy service (and its developers) would need to add support for the ACME protocol. For any new protocols Vault supports in the future, this service would also need to implement support to retain compatibility.

Custom policy with Certificate Issuance External Policy Service (CIEPS)

With CIEPS, users still authenticate to Vault and use the normal request workflow to sign and issue certificates, including via ACME. However, Vault's PKI Engine reaches out to the configured CIEPS implementation to validate and template the requested certificate, transparently from the calling application.

Notably, the application can opt to either retain full control over its key material or delegate key creation to the trusted Vault service, with no impact on the functionality CIEPS can provide. The CIEPS service can be scoped to respond to requests from either a single PKI mount or multiple, getting information about the requesting user and the Vault PKI instance from the CIEPS messages.

Because the CIEPS service only needs knowledge of validating requests and templating the final certificate structure, its developers need only be concerned with the business policy logic and not broader PKI concerns (such as generating key material or re-implementing support for other issuance protocols).

Certificate Issuance External Policy Service (CIEPS) webhook format

The CIEPS protocol is a REST-based, optionally mTLS protected webhook. The external service configuration specifies the single URL that Vault will POST the formatted CIEPS request to. When the CIEPS service is unavailable (either due to misconfiguration or outage), Vault will reject the request and it is up to the client to retry the request at a later time.

For convenience, Go versions of these structs are available from the Vault SDK.

Vault to CIEPS request format

This document outlines CIEPS request/response version 1.

Using the application/json content type, Vault will post the following request body as a JSON object:

request_version (int: 1) - The version of the CIEPS request sent by Vault; a compatible response format is expected.
request_uuid (string) - A random UUID which serves to identify this request. This value must be sent in the response.
synchronous (bool: true) - A boolean indicating whether the request is synchronous or not. Presently set to true; no asynchronous response is understood.
user_request_key_values (map[string]interface{}) - The unvalidated request parameters sent by the user. It is up to the CIEPS service to validate these prior to using them. The following fields may be present, including any other fields submitted by the user:
- csr (string) - A PEM format CSR submitted either by the client ( in the case of /sign or ACME requests) or on the client's behalf (in the case of /issue requests, where key material is generated by Vault).
identity_request_key_values (map[string]interface{}) - Values related to the user's identity. When the request type is ACME, this value is not populated. These are:
- entity_id (string) - The entity identifier from the request after authentication.
- entity (map[string]interface{}) - The entire resolved logical.Entity of the user after authentication; subject to change by the entity_jmsepath parameter in the configuration.
- groups ([]map[string]interface{}) - The set of resolved logical.Groups of the user after authentication; subject to change by the group_jmsepath parameter in the configuration.
Note: in the event that the direct token backend or a root token is used, entity information may not exist. In either case, identity_request_key_values will be omitted.
acme_request_key_values (map[string]interface{}) - Values related to ACME authorizations challenges attached to the finished order. Only present when the request type is ACME. These are:
- authorizations (map[string]interface{}) - Authorizations and challenges solved by the client to move this order to the finalization state.
- account (map[string]interface{}) - Information related to the ACME account issuing the request. These are:
  - id (string) - The UUID of the ACME account.
  - directory (string) - The path to the ACME directory requested by this account.
  - contact ([]string) - Unverified contact information submitted by the requesting ACME account on creation.
  - created_date (string: RFC 3999 format) - Timestamp when the account was created.
  - eab (map[string]interface{}, optional) - When present, the details of the EAB used to authorize this account via Vault authentication. If not present, this ACME account was created without EAB bindings.
    - key_id (string) - Identifier of the EAB binding used by this account.
    - key_type (string) - Key type of the EAB binding used by this account.
    - created_date (string: RFC 3999 format) - Timestamp when the account was created.
vault_request_values (map[string]interface{}) - Request value validated or created by Vault. These have higher trust than the unvalidated user_request_key_values. These are:
- policy_name (string: "") - The optional policy name specified by the requester. When the issuance mode is not ACME (or if it was ACME and EAB was enforced), this has been validated by Vault's ACL system.
- mount (string) - The request's mount path as known by the PKI plugin.
- namespace (string) - The request's namespace the mount path exists within as known by the PKI plugin.
- vault_is_performance_standby (bool) - Asserted when this requesting node is a standby node. When the service indicates storage is required in its response, Vault will forward the user's HTTP request up to an active node, requiring it to re-submit the CIEPS request. In this case, if the service knows it must always store certificates and sees a request from a standby node, it can skip policy and template evaluation or cache the results for a second pass.
- vault_is_performance_secondary (bool) - Asserted when this requesting node is from a performance secondary versus the primary cluster.
- issuance_mode (string: "sign", "issue", "ica", or "acme") - The type of the request: whether a REST call to /external-policy/sign(/:policy), to /external-policy/issue(/:policy), /external-policy/sign-intermediate(/:policy), or an ACME request, respectively.
- vault_generated_private_key (bool) - Whether or not Vault generated the key material behind this request. Set to true when issuance_mode="issue" only presently.
- requested_issuer_name (string) - Name of the user's requested issuer; can be changed by modifying the response issuer_ref value.
- requested_issuer_id (string) - UUID of the user's requested issuer; can be changed by modifying the response issuer_ref value.
- requested_issuer_cert (string) - PEM format certificate of the user's requested issuer; can be changed by modifying the response issuer_ref value.
- requested_issuance_config (map[string]interface{}) - Configuration used for leaf certificate issuance. These are:
  - aia_values (map[string]interface{}) - AIA values (CA, CRL, and OCSP) for the suggested issuer. These may differ from the actual values used for issuance of this request if issuer_ref is set on the response.
  - leaf_not_after_behavior (string: "err", "truncate", or "permit") - leaf validity period behavior for the suggested issuer.
  - mount_default_ttl (string) - Suggested default TTL set on mount tuning.
  - mount_max_ttl (string) - Suggested maximum TTL set on the mount tuning.

CIEPS to Vault response format

The CIEPS engine must reply to this POST response with a 200 OK status, regardless of whether a certificate should be issued or not. Redirects will not be followed by Vault; any proxy or load balancing functionality should be strictly transparent to the caller. Any verbatim message returned by a non-200 status code will not be returned, either in Vault server logs or to the user.

In the response to the above request, only one of the certificate or error fields should be specified. In the event both certificate and error are present, the error will be appended to the returned warnings and the certificate will be issued.

Using the application/json content type, the server should reply with the following JSON object:

request_uuid (string) - The random UUID which the server used to identify this request.
error (string, optional) - The error message to be returned to the user about why their request failed. Only one of the error or certificate response parameters should be specified.
warnings ([]string, optional) - Optional warnings to be returned to the user about minor issues with their request.
certificate (string, optional) - A PEM format certificate to be signed by the Vault service. Only one of the error or certificate response parameters should be specified.
issuer_ref (string) - The issuer reference to use to sign this request. If the user's issuer choice (in requested_issuer_id) is OK, this must be set in this field.
store_certificate (bool: false) - Whether or not to store the signed certificate.
generate_lease (bool: false) - Whether or not Vault should generate an associated lease for the certificate. Note that to generate a lease, store_certificate also needs to be set to true, otherwise no lease will be generated.

The certificate's signature will be ignored and replaced by a signature created by the specified issuer. If a signature algorithm compatible with this issuer is specified on the certificate, it will be preserved; otherwise, the default signature algorithm for this issuer's key type will be used.

The certificate's AIA information will be replaced by the information from the specified issuer, if present, else the global AIA URLs will be set, replacing the AIA URIs and CRL distribution point extensions. Additionally, the Authority Key Identifier extension will be replaced by the issuer's Subject Key Identifier extension value as mandated by RFC 5280.

Tutorial

Refer to the following tutorials for PKI secrets engine usage examples:

API

The PKI secrets engine has a full HTTP API. Please see the PKI secrets engine API for more details.