HashiCorp Developer AI Private beta now available Sign up today
Terraform
Terraform Home

You are viewing documentation for version v202401-2. View latest version.

Run States and Stages

Each run passes through several stages of action (pending, plan, cost estimation, policy check, apply, and completion), and Terraform Cloud shows the progress through those stages as run states.

In the list of workspaces on Terraform Cloud's main page, each workspace shows the state of the run it's currently processing. (Or, if no run is in progress, the state of the most recent completed run.)

The Pending Stage

States in this stage:

  • Pending: Terraform Cloud hasn't started action on a run yet. Terraform Cloud processes each workspace's runs in the order they were queued, and a run remains pending until every run before it has completed.

Leaving this stage:

  • If the user discards the run before it starts, the run does not continue (Discarded state).
  • If the run is first in the queue, it proceeds automatically to the plan stage (Planning state).

The Fetching Stage

Terraform Cloud may need to fetch the configuration from VCS prior to starting the plan. Terraform Cloud automatically archives configuration versions created through VCS when all runs are complete and then re-fetches the files for subsequent runs.

States in this stage:

  • Fetching: If Terraform Cloud has not yet fetched the configuration from VCS, the run will go into this state until the configuration is available.

Leaving this stage:

  • If Terraform Cloud encounters an error when fetching the configuration from VCS, the run does not continue (Plan Errored state).
  • If Terraform successfully fetches the configuration, the run moves to the next stage.

The Pre-Plan Stage

The pre-plan phase only occurs if there are enabled run tasks in the workspace that are configured to begin before Terraform creates the plan. Terraform Cloud sends information about the run to the configured external system and waits for a passed or failed response to determine whether the run can continue. The information sent to the external system includes the configuration version of the run.

All runs can enter this phase, including speculative plans.

States in this stage:

  • Pre-plan running: Terraform Cloud is waiting for a response from the configured external system(s).
    • External systems must respond initially with a 200 OK acknowledging the request is in progress. After that, they have 10 minutes to return a status of passed, running, or failed. If the timeout expires, Terraform Cloud assumes that the run tasks is in the failed status.

Leaving this stage:

  • If any mandatory tasks failed, the run skips to completion (Plan Errored state).
  • If any advisory tasks failed, the run proceeds to the Planning state, with a visible warning regarding the failed task.
  • If a single run has a combination of mandatory and advisory tasks, Terraform takes the most restrictive action. For example, the run fails if there are two advisory tasks that succeed and one mandatory task that fails.
  • If a user canceled the run, the run ends in the Canceled state.

The Plan Stage

A run goes through different steps during the plan stage depending on whether or not Terraform Cloud needs to fetch the configuration from VCS. Terraform Cloud automatically archives configuration versions created through VCS when all runs are complete and then re-fetches the files for subsequent runs.

States in this stage:

  • Planning: Terraform Cloud is currently running terraform plan.
  • Needs Confirmation: terraform plan has finished. Runs sometimes pause in this state, depending on the workspace and organization settings.

Leaving this stage:

  • If the terraform plan command failed, the run does not continue (Plan Errored state).
  • If a user canceled the plan by pressing the "Cancel Run" button, the run does not continue (Canceled state).
  • If the plan succeeded with no changes and neither cost estimation nor Sentinel policy checks will be done, Terraform Cloud considers the run complete (Planned and Finished state).
  • If the plan succeeded and requires changes:
    • If cost estimation is enabled, the run proceeds automatically to the cost estimation stage.
    • If cost estimation is disabled and [Sentinel policies][] are enabled, the run proceeds automatically to the policy check stage.
    • If there are no Sentinel policies and the plan can be auto-applied, the run proceeds automatically to the apply stage. Plans can be auto-applied if the auto-apply setting is enabled on the workspace and the plan was queued by a new VCS commit or by a user with permission to apply runs. (More about permissions.)
    • If there are no Sentinel policies and Terraform Cloud cannot auto-apply the plan, the run pauses in the Needs Confirmation state until a user with permission to apply runs takes action. (More about permissions.) If an authorized user approves the apply, the run proceeds to the apply stage. If an authorized user rejects the apply, the run does not continue (Discarded state).

Note, if you want to directly integrate third-party tools and services between your plan and apply stages, see Run Tasks.

The Post-Plan Stage

The post-plan phase executes any configured run tasks after the plan is complete, so it only occurs if there are run tasks enabled for the workspace. All runs can enter this phase, including speculative plans. During this phase, Terraform Cloud sends information about the run to the configured external system and waits for a passed or failed response to determine whether the run can continue.

Note: The information sent to the configured external system includes the JSON output of the Terraform plan.

States in this stage:

  • Post-plan running: Terraform Cloud is waiting for a response from the configured external system(s).
    • External systems must respond initially with a 200 OK acknowledging the request is in progress. After that, they have 10 minutes to return a status of passed, running, or failed, or the timeout will expire and the task will be assumed to be in the failed status.

Leaving this stage:

  • If any mandatory tasks failed, the run skips to completion (Plan Errored state).
  • If any advisory tasks failed, the run proceeds to the Applying state, with a visible warning regarding the failed task.
  • If a single run has a combination of mandatory and advisory tasks, Terraform takes the most restrictive action. For example, if there are two advisory tasks that succeed and one mandatory task that failed, the run fails. If one mandatory task succeeds and two advisory tasks fail, the run succeeds with a warning.
  • If a user canceled the run, the run ends in the Canceled state.

The OPA Policy Check Stage

This stage only occurs if you enabled Open Policy Agent (OPA) policies and runs after a successful terraform plan and before Cost Estimation. In this stage, Terraform Cloud checks whether the plan adheres to the policies in the OPA policy sets for the workspace.

States in this stage:

  • Policy Check: Terraform Cloud is checking the plan against the OPA policy sets.
  • Policy Override: The policy check finished, but a mandatory policy failed. The run pauses, and Terraform cannot perform an apply unless a user manually overrides the policy check failure. Refer to Policy Results for details.
  • Policy Checked: The policy check succeeded, and Terraform can apply the plan. The run may pause in this state if the workspace is not set up to auto-apply runs.

Leaving this stage:

If any mandatory policies failed, the run pauses in the Policy Override state. The run completes one of the following workflows:

  • The run stops and enters the Discarded state when a user with permission to apply runs discards the run.
  • The run proceeds to the Policy Checked state when a user with permission to manage policy overrides overrides the failed policy. The Policy Checked state means that no mandatory policies failed or that a user performed a manual override.

Once the run reaches the Policy Checked state, the run completes one of the following workflows:

  • The run proceeds to the Apply stage if Terraform can automatically apply the plan. An auto-apply requires that the Auto apply setting is enabled on the workspace.
  • If Terraform cannot automatically apply the plan, the run pauses in the Policy Checked state until a user with permission to apply runs takes action. If the user approves the apply, the run proceeds to the Apply stage. If the user rejects the apply, the run stops and enters the Discarded state.

The Cost Estimation Stage

This stage only occurs if cost estimation is enabled. After a successful terraform plan, Terraform Cloud uses plan data to estimate costs for each resource found in the plan.

States in this stage:

  • Cost Estimating: Terraform Cloud is currently estimating the resources in the plan.
  • Cost Estimated: The cost estimate completed.

Leaving this stage:

  • If cost estimation succeeded or errors, the run moves to the next stage.
  • If there are no policy checks or applies, the run does not continue (Planned and Finished state).

The Sentinel Policy Check Stage

This stage only occurs if Sentinel policies are enabled. After a successful terraform plan, Terraform Cloud checks whether the plan obeys policy to determine whether it can be applied.

States in this stage:

  • Policy Check: Terraform Cloud is currently checking the plan against the organization's policies.
  • Policy Override: The policy check finished, but a soft-mandatory policy failed, so an apply cannot proceed without approval from a user with permission to manage policy overrides for the organization. (More about permissions.) The run pauses in this state.
  • Policy Checked: The policy check succeeded, and Sentinel will allow an apply to proceed. The run sometimes pauses in this state, depending on workspace settings.

Leaving this stage:

  • If any hard-mandatory policies failed, the run does not continue (Plan Errored state).
  • If any soft-mandatory policies failed, the run pauses in the Policy Override state.
    • If a user with permission to manage policy overrides, overrides the failed policy, the run proceeds to the Policy Checked state. (More about permissions.)
    • If a user with permission to apply runs discards the run, the run does not continue (Discarded state). (More about permissions.)
  • If the run reaches the Policy Checked state (no mandatory policies failed, or soft-mandatory policies were overridden):
    • If the plan can be auto-applied, the run proceeds automatically to the apply stage. Plans can be auto-applied if the auto-apply setting is enabled on the workspace and the plan was queued by a new VCS commit or by a user with permission to apply runs. (More about permissions.)
    • If the plan can't be auto-applied, the run pauses in the Policy Checked state until a user with permission to apply runs takes action. (More about permissions.) The run proceeds to the apply stage if they approve the apply, or does not continue (Discarded state) if they reject the apply.

The Pre-Apply Stage

The pre-apply phase only occurs if the workspace has run tasks configured to begin before Terraform creates the apply. Terraform Cloud sends information about the run to the configured external system and waits for a passed or failed response to determine whether the run can continue. The information sent to the external system includes the configuration version of the run.

Only confirmed runs can enter this phase.

States in this stage:

  • Pre-apply running: Terraform Cloud is waiting for a response from the configured external system(s).
    • External systems must respond initially with a 200 OK acknowledging the request is in progress. After that, they have 10 minutes to return a status of passed, running, or failed. If the timeout expires, Terraform Cloud assumes that the run tasks is in the failed status.

Leaving this stage:

  • If any mandatory tasks failed, the run skips to completion.
  • If any advisory tasks failed, the run proceeds to the Applying state, with a visible warning regarding the failed task.
  • If a single run has a combination of mandatory and advisory tasks, Terraform takes the most restrictive action. For example, the run fails if there are two advisory tasks that succeed and one mandatory task that fails.
  • If a user canceled the run, the run ends in the Canceled state.

The Apply Stage

States in this stage:

  • Applying: Terraform Cloud is currently running terraform apply.

Leaving this stage:

After applying, the run proceeds automatically to completion.

  • If the apply succeeded, the run ends in the Applied state.
  • If the apply failed, the run ends in the Apply Errored state.
  • If a user canceled the apply by pressing Cancel Run, the run ends in the Canceled state.

The Post-Apply Stage

The post-apply phase only occurs if the workspace has run tasks configured to begin after Terraform complete the apply operation.

Unlike other run task stages, the post-apply stage gathers information about the resulting response of an apply operation because that operation has already occurred, and Terraform Cloud has already provisioned infrastructure. Terraform Cloud sends the information about the run to the configured external system with the passed or failed status along with the configuration version of the run.

Only confirmed runs can enter this phase.

States in this stage:

  • Post-apply running: Terraform Cloud is waiting for a response from the configured external system(s).
  • External systems must respond initially with a 200 OK acknowledging the request is in progress. After that, they have 10 minutes to return a status of passed, running, or failed. If the timeout expires, Terraform Cloud assumes that the run tasks is in the failed status.

Leaving this stage:

  • There are only advisory tasks on this stage.
  • If any advisory tasks failed, the run proceeds to the Applied state, with a visible warning regarding the failed task.
  • If a user cancels the run, the run ends in the Canceled state.

Completion

A run is complete if it finishes applying, if any part of the run fails, if there is nothing to do, or if a user chooses not to continue. Once a run completes, the next run in the queue can enter the plan stage.

States in this stage:

  • Applied: The run was successfully applied.
  • Planned and Finished: terraform plan's output already matches the current infrastructure state, so terraform apply doesn't need to do anything.
  • Apply Errored: The terraform apply command failed, possibly due to a missing or misconfigured provider or an illegal operation on a provider.
  • Plan Errored: The terraform plan command failed (usually requiring fixes to variables or code), or a hard-mandatory Sentinel policy failed. The run cannot be applied.
  • Discarded: A user chose not to continue this run.
  • Canceled: A user interrupted the terraform plan or terraform apply command with the "Cancel Run" button.