Health

Terraform Cloud can perform automatic health assessments in a workspace to assess whether its real infrastructure matches the requirements defined in its Terraform configuration. Health assessments include the following types of evaluations:

• Drift detection determines whether your real-world infrastructure matches your Terraform state file.
• Continuous validation determines whether custom conditions in the workspace’s configuration continue to pass after Terraform provisions the infrastructure.

When enabled, Terraform Cloud automatically runs a health assessment for your workspace about every 24 hours. Refer to Health Assessment Scheduling for details.

Permissions

Working with health assessments requires the following permissions:

• To view health status for a workspace, you need read access to that workspace.
• To change organization health settings, you must be an organization owner.
• To change a workspace’s health settings, you must be an administrator for that workspace.

Workspace Requirements

Workspaces require the following settings to receive health assessments:

• Terraform version 0.15.4+ for drift detection only
• Terraform version 1.3.0+ for drift detection and continuous validation
• Remote execution mode or Agent execution mode for Terraform runs

The latest Terraform run in the workspace must have been successful. If the most recent run ended in an errored, canceled, or discarded state, Terraform Cloud pauses health assessments until there is a successfully applied run.

The workspace must also have at least one run in which Terraform successfully applies a configuration. Terraform Cloud does not perform health assessments in workspaces with no real-world infrastructure.

Enable Health Assessments

You can enforce health assessments across all eligible workspaces in an organization within the organization settings. Enforcing health assessments at an organization-level overrides workspace-level settings. You can only enable health assessments within a specific workspace when Terraform Cloud is not enforcing health assessments at the organization level.

To enable health assessments within a workspace:

1. Verify that your workspace satisfies the requirements.
2. Go to the workspace and click Settings > Health.
3. Select Enable under Health Assessments.
4. Click Save settings.

Health Assessment Scheduling

The timing of the first health assessment in the workspace depends on whether you enable health assessments during active Terraform runs:

• No active runs: The first health assessment starts a few minutes after you enable the feature.
• Active speculative plan: The first health assessment starts soon after that plan's completion.
• Other active runs: The first health assessment starts in about 24 hours.

After the first health assessment, Terraform Cloud starts a new health assessment if at least 24 hours have passed since the last assessment and there are no active runs in the workspace. Health assessments may take longer to complete when you enable health assessments in many workspaces at once or your workspace contains a complex configuration with many resources.

A health assessment never interrupts or interferes with runs. If you start a new run during a health assessment, Terraform Cloud cancels the current assessment and runs the next assessment in 24 hours. This behavior may prevent Terraform Cloud from performing health assessments in workspaces with frequent runs.

Terraform Cloud pauses health assessments if the latest run ended in an errored state. This behavior occurs for all run types, including plan-only runs and speculative plans. Once the workspace completes a successful run, Terraform Cloud restarts health assessments after 24 hours.

Terraform Enterprise administrators can modify their installation's assessment frequency and number of maximum concurrent assessments from the admin settings console.

Concurrency

If you enable health assessments on multiple workspaces, assessments may run concurrently. Health assessments do not affect your concurrency limit. Terraform Cloud also monitors and controls health assessment concurrency to avoid issues for large-scale deployments with thousands of workspaces. However, Terraform Cloud performs health assessments in batches, so health assessments may take longer to complete when you enable them in a large number of workspaces.

Notifications

Terraform Cloud sends notifications about health assessment results according to your workspace’s settings.

Workspace Health Status

On the organization's Workspaces page, Terraform Cloud displays a Health warning status for workspaces with infrastructure drift or failed continuous validation checks.

On the right of a workspace’s overview page, Terraform Cloud displays a Health bar that summarizes the results of the last health assessment.

• The Drift summary shows the total number of resources in the configuration and the number of resources that have drifted.
• The Checks summary shows the number of passed, failed, and unknown statuses for objects with continuous validation checks.

Drift Detection

Infrastructure drift means that your real-world infrastructure no longer matches your Terraform state file. Drift occurs when a user modifies resources outside of the Terraform workflow. For example, a colleague may update resource configuration directly in the cloud provider console to resolve a production incident. This action changes the real resource attributes from those tracked in the state file.

View Workspace Drift

To view the continuous validation results from the latest health assessment, go to the workspace and click Health > Drift. If there is drift, Terraform Cloud shows how the real infrastructure differs from the latest version of the workspace’s state file.

Resolve Drift

You can use one of the following approaches to correct workspace drift:

• Overwrite drift: Queue a new plan to realign your real-world infrastructure with your Terraform configuration.
• Update Terraform state and configuration: Queue a refresh-only plan to update your Terraform state to match your real-world infrastructure. We recommend also modifying your Terraform configuration to include any new or changed resources. Otherwise, Terraform will overwrite the updated state file during the next apply. Refer to our Manage Resource Drift tutorial for a detailed example.

Continuous Validation

The Terraform configuration language provides precondition and postcondition blocks to create custom rules for resources, data sources, and outputs. These rules help validate your configuration.

Continuous validation lets Terraform Cloud regularly check whether the preconditions and postconditions in a workspace’s configuration continue to pass, validating the real-world infrastructure. For example, you can write a postcondition to check whether an API gateway certificate is valid. Continuous validation alerts you when the condition fails, so you can update the certificate and avoid errors the next time you want to update your infrastructure.

Refer to Preconditions and Postconditions for more details about adding custom conditions in your Terraform configuration.

Example Use Cases

HCP Packer stores metadata about your Packer images. The following example postcondition fails when there is a newer AMI version available.

resource "aws_instance" "hashiapp" {
  ami                         = data.hcp_packer_image.hashiapp_image.cloud_image_id
  instance_type               = var.instance_type
  associate_public_ip_address = true
  subnet_id                   = aws_subnet.hashiapp.id
  vpc_security_group_ids      = [aws_security_group.hashiapp.id]
  key_name                    = aws_key_pair.generated_key.key_name

  lifecycle {
    postcondition {
      condition     = self.ami == data.hcp_packer_image.hashiapp_image.cloud_image_id
      error_message = "Must use the latest available AMI,
        ${data.hcp_packer_image.hashiapp_image.cloud_image_id}."
    }
  }
}

Vault lets you secure, store, and tightly control access to tokens, passwords, certificates, encryption keys, and other sensitive data. The following example postcondition fails when a Vault certificate expires.

resource "vault_pki_secret_backend_cert" "app" {
  backend = vault_mount.intermediate.path
  name = vault_pki_secret_backend_role.test.name
  common_name = "app.my.domain"

  lifecycle {
    postcondition {
      condition = !self.renew_pending
      error_message = "Vault cert is ready to renew."
    }
  }
}

View Continuous Validation Checks

To view the continuous validation results from the latest health assessment, go to the workspace and click Health > Continuous validation. The page shows all of the resources, outputs, and data sources with custom conditions that Terraform Cloud checked. Next to each object, Terraform Cloud reports whether the checks passed or failed. If one or more checks failed, Terraform Cloud displays the error messages for those conditions.

A single resource, output, or data source may have multiple preconditions or postconditions. Terraform Cloud does not show the results from individual conditions unless they fail. If all custom conditions on the object pass, Terraform Cloud reports that the entire check passed.