Acceptance Tests
Tip
Terraform Provider testing has migrated to a new terraform-plugin-testing Go module. New features will only be implemented in the new module. Review the migration documentation to learn how to switch existing testing code.
In order to deliver on our promise to be safe and predictable, we need to be able to easily and routinely verify that Terraform Plugins produce the expected outcome. The most common usage of an acceptance test is in Terraform Providers, where each Resource is tested with configuration files and the resulting infrastructure is verified. Terraform includes a framework for constructing acceptance tests that imitate the execution of one or more steps of applying one or more configuration files, allowing multiple scenarios to be tested.
Terraform acceptance tests use real Terraform configurations to exercise the code in real plan, apply, refresh, and destroy life cycles. When run from the root of a Terraform Provider codebase, Terraform’s testing framework compiles the current provider in-memory and executes the provided configuration in developer defined steps, creating infrastructure along the way. At the conclusion of all the steps, Terraform automatically destroys the infrastructure. It’s important to note that during development, it’s possible for Terraform to leave orphaned or “dangling” resources behind, depending on the correctness of the code in development. The testing framework provides means to validate all resources are destroyed, alerting developers if any fail to destroy. It is the developer's responsibility to clean up any dangling resources left over from testing and development.
How Acceptance Tests Work
Provider acceptance tests use a Terraform CLI binary to run real Terraform commands. The goal is to approximate using the provider with Terraform in production as closely as possible.
Terraform Core and Terraform Plugins act as gRPC client and server, implemented using HashiCorp's go-plugin system (refer to the RPC Plugin Model section of the Terraform documentation). When go test
is run, the SDK's acceptance test framework starts a plugin server in the same process as the Go test framework. This plugin server runs for the duration of the test case, and each Terraform command (terraform plan
, terraform apply
, etc) creates a client that reattaches to this server.
Real-world Terraform usage requires a config file and Terraform working directory on the local filesystem. The framework uses the internal/plugintest
package to manage temporary directories and files during test runs. Provider developers should not use this library directly.
While the test framework provides a reasonable simulation of real-world usage, there are some differences, the major one being in the lifecycle of the plugin gRPC server. During normal Terraform operation, the plugin server starts and stops once per graph walk, of which there may be several during one Terraform command. The acceptance test framework, however, maintains one plugin gRPC server for the duration of each test case. In theory, it is possible for providers to carry internal state between operations during tests - but providers would have to go out of their way (and the SDK's public API) to do this.
Test files
Terraform follows many of the Go programming language conventions with regards
to testing, with both acceptance tests and unit tests being placed in a file
that matches the file under test, with an added _test.go
suffix. Here’s an
example file structure:
To create an acceptance test in the example resource_example_compute_test.go
file, the function name must begin with TestAccXxx
, and have the following
signature:
Requirements and Recommendations
Acceptance tests have the following requirements:
- Go: The most recent stable version.
- Terraform CLI: Version 0.12.26 or later.
- Provider Access: Network or system access to the provider and any resources being tested.
- Provider Credentials: Authorized credentials to the provider and any resources being tested.
- TF_ACC Environment Variable: Set to any value. Prevents developers from incurring unintended charges when running other Go tests.
We also recommend the following when running acceptance tests:
- Separate Account: Use a separate provider account or namespace for acceptance testing. This prevents Terraform from unexpectedly modifying or destroying infrastructure due to code or testing issues.
- Previous Terraform CLI Installation: Install Terraform CLI either into the operating system
PATH
or use theTF_ACC_TERRAFORM_PATH
environment variable prior to running acceptance tests. Otherwise, the testing framework will download and install the latest Terraform CLI version into a temporary directory for every test invocation. Refer to the Terraform CLI Installation Behaviors section for details.
Each provider may have additional requirements and setup recommendations. Refer to the provider's codebase for more details.
Terraform CLI Installation Behaviors
The testing framework implements the following Terraform CLI discovery and installation behaviors:
- If the
TF_ACC_TERRAFORM_PATH
environment variable is set, the framework will use that Terraform CLI binary if it exists and is executable. If the framework cannot find the binary or it is not executable, the framework returns an error unless theTF_ACC_TERRAFORM_VERSION
environment variable is also set. - If the
TF_ACC_TERRAFORM_VERSION
environment variable is set, the framework will install and use that Terraform CLI version. - If both the
TF_ACC_TERRAFORM_PATH
andTF_ACC_TERRAFORM_VERSION
environment variables are unset, the framework will search for the Terraform CLI binary based on the operating systemPATH
. If the framework cannot find the specified binary, it installs the latest available Terraform CLI binary.
Refer to the Environment Variables section for more details about behaviors and valid configurations.
Running Acceptance Tests
Ensure that the acceptance testing requirements are met and then use the go test
command to run acceptance tests. You can run the acceptance tests on any environment capable of running go test
, such as a local workstation command line, or continuous integration runner, such as GitHub Actions.
Note: Acceptance tests typically create and destroy actual infrastructure resources, possibly incurring expenses during or after the test duration.
Command Line Workflow
Run acceptance testing with the command line of any workstation. Use these instructions as the basis for other environments such as continuous integration runners.
The following example will execute all available acceptance tests in a provider codebase:
Some provider codebases also implement a Makefile with a testacc
target, which will set TF_ACC
and other testing flags automatically.
The following is an example Makefile configuration:
The Makefile configuration lets developers to use the following command to run acceptance tests:
GitHub Actions Workflow
If using GitHub, run acceptance testing via GitHub Actions. Other continuous integration runners, while not exhaustively documented, are also supported.
Ensure the GitHub Organization settings for GitHub Actions and GitHub Repository settings for GitHub Actions allows running workflows and allows the actions/checkout
, actions/setup-go
, and hashicorp/setup-terraform
actions.
Create a GitHub Actions workflow file, such as .github/workflows/test.yaml
, that does the following:
- Runs when pull requests are submitted or on other events as appropriate.
- Uses
actions/checkout
to checkout the provider codebase. - Uses
actions/setup-go
to install Go. - Uses
hashicorp/setup-terraform
to install Terraform CLI. - Runs the
go test
command with the appropriate environment variables and flags.
Use the matrix
strategy for more advanced configuration, such as running acceptance testing against multiple Terraform CLI versions.
The following example workflow runs acceptance testing for the provider using the latest patch versions of Go defined in the go.mod
file and Terraform CLI 1.5:
The following example workflow runs acceptance testing for the provider using the latest patch versions of Go defined in the go.mod
file and Terraform CLI 0.12 through 1.5:
Environment Variables
A number of environment variables are available to control aspects of acceptance test execution.
Environment Variable Name | Default | Description |
---|---|---|
TF_ACC | N/A | Set to any value to enable acceptance testing via the helper/resource.ParallelTest() and helper/resource.Test() functions. |
TF_ACC_PROVIDER_HOST : | registry.terraform.io | Set the hostname of the provider under test, such as example.com in the example.com/myorg/myprovider provider source address. This is only required if any TestStep.Config specifies a provider source address, such as in the terraform configuration block required_providers attribute. |
TF_ACC_PROVIDER_NAMESPACE | hashicorp | Set the namespace of the provider under test, such as myorg in the registry.terraform.io/myorg/myprovider provider source address. This is only required if any TestStep.Config specifies a provider source address, such as in the terraform configuration block required_providers attribute. |
TF_ACC_STATE_LINEAGE | N/A | Set to 1 to enable state lineage debug logs, which are normally suppressed during acceptance testing. |
TF_ACC_TEMP_DIR | Operating system specific via os.TempDir() | Set a temporary directory used for testing files and installing Terraform CLI, if installation is required. |
TF_ACC_TERRAFORM_PATH | N/A | Set the path to a Terraform CLI binary on the local filesystem to be used during testing. It must be executable. If not found and TF_ACC_TERRAFORM_VERSION is not set, an error is returned. |
TF_ACC_TERRAFORM_VERSION | N/A | Set the exact version of Terraform CLI to automatically install into TF_ACC_TEMP_DIR . For example, 1.1.6 or v1.0.11 . |
Logging Environment Variables
A number of environment variables available to control logging aspects during acceptance test execution. Some of these modify or replace the production behaviors defined in managing provider log output and debugging Terraform.
Logging Levels
Environment Variable Name | Default | Description |
---|---|---|
TF_ACC_LOG | N/A | Set the TF_LOG environment variable used by Terraform CLI while testing. If set, overrides TF_LOG_CORE . Use TF_LOG_CORE and TF_LOG_PROVIDER to configure separate levels for Terraform CLI logging. |
TF_LOG | N/A | Set the log level for the Go standard library log package. If set to any level, sets the TRACE log level for any SDK and provider logs written by terraform-plugin-log . Use the TF_LOG_SDK* and TF_LOG_PROVIDER_* environment variables described in managing log output to decrease or disable SDK and provider logs written by terraform-plugin-log . Use TF_ACC_LOG , TF_LOG_CORE , or TF_LOG_PROVIDER environment variables to set the logging levels used by Terraform CLI while testing. |
TF_LOG_CORE | TF_ACC_LOG | Set the TF_LOG_CORE environment variable used by Terraform CLI logging of graph operations and other core functionality while testing. If TF_ACC_LOG is set, this setting has no effect. Use TF_LOG_PROVIDER to configure a separate level for Terraform CLI logging of external providers while testing (e.g. defined by the TestCase or TestStep type ExternalProviders field). |
TF_LOG_PROVIDER | TF_ACC_LOG | Set the TF_LOG_PROVIDER environment variable used by Terraform CLI logging of external providers while testing (e.g. defined by the TestCase or TestStep type ExternalProviders field). If set, overrides TF_ACC_LOG . Use TF_LOG_CORE to configure a separate level for Terraform CLI logging of graph operations and other core functionality while testing. |
Logging Output
By default, there is no logging output when running the go test
command. Use one of the below environment variables to output logs to the local filesystem or use the go test
command -v
(verbose) flag to view logging without writing file(s).
Environment Variable Name | Default | Description |
---|---|---|
TF_ACC_LOG_PATH | N/A | Set a file path for all logs during testing. Use TF_LOG_PATH_MASK to configure individual log files per test. |
TF_LOG_PATH_MASK | N/A | Set a file path containing the string %s , which is replaced with the test name, to write a separate log file per test. Use TF_ACC_LOG_PATH to configure a single log file for all tests. |
The logs associated with each test can output across incorrect files as each new test starts if the provider is using the Go standard library log
package for logging, acceptance testing that uses helper/resource.ParallelTest()
, and TF_LOG_PATH_MASK
. To resolve this issue, choose one of the following approaches:
- Use
terraform-plugin-log
based logging. Each logger will be correctly associated with each test name output. - Wrap testing execution so that each test is individually executed with
go test
. Since eachgo test
process will have its ownlog
package output handling, logging will be correctly associated with each test name output. - Replace
helper/resource.ParallelTest()
withhelper/resource.Test()
and ensure(*testing.T).Parallel()
is not called in tests. This serializes all testing so each test will be associated with each test name output.
Troubleshooting
This section lists common errors encountered during testing.
Unrecognized remote plugin message
This error indicates that the provider server could not connect to Terraform Core. Verify that the output of terraform version
is v0.12.26 or above.
Next Steps
Terraform relies heavily on acceptance tests to ensure we keep our promise of helping users safely and predictably create, change, and improve infrastructure. In our next section we detail how to create “Test Cases”, individual acceptance tests using Terraform’s testing framework, in order to build and verify real infrastructure. Proceed to Test Cases