Implement a provider with the Terraform Plugin Framework

11min
|
Terraform

In these tutorials, you will write a custom Terraform provider against the API of a fictional coffee-shop application called HashiCups using the Terraform Plugin Framework. You will learn how to create data sources, authenticate the provider to the HashiCups client, and how providers map target APIs to Terraform in order to create, read, update, and delete resources. You will also implement testing, documentation generation, and learn how to publish your providers to the Terraform Registry.

There are a several reasons to author a custom provider, including:

Enable users of your product to provision components with Terraform.
Enable users of internal products and services to provision components with Terraform.
Extend the capabilities of an existing provider by fixing bugs or adding new features and customization options.

In this tutorial, you will set up your Terraform provider development environment and create an initial provider that can communicate with Terraform. To do this, you will:

Set up your development environment.
Clone the terraform-provider-scaffolding-framework repository and add additional files needed for testing. This repo contains a scaffold for a generic Terraform provider.
Implement the provider type.
Prepare the provider code for all future implementation details, such as implementing data sources and resources.
Implement the provider server.
Create a provider server to allow Terraform core to connect with and use your provider.
Install and verify the provider locally.
This allows you to manually test your provider during development.

Prerequisites

To follow this tutorial, you will need:

Go 1.21+ installed and configured.
Terraform v1.8+ installed locally.
Docker and Docker Compose to run an instance of HashiCups locally.

Set up your development environment

Clone the Terraform Provider Scaffolding Framework repository.

$ git clone https://github.com/hashicorp/terraform-provider-scaffolding-framework

We recommend that you use this scaffolding repository as a starting point for any new providers you create. It contains a template for a new provider that you will extend and customize as you develop your provider.

Rename the directory to terraform-provider-hashicups.

$ mv terraform-provider-scaffolding-framework terraform-provider-hashicups

Change into the cloned repository.

$ cd terraform-provider-hashicups

Rename the go.mod module.

$ go mod edit -module terraform-provider-hashicups

Then, install all the provider's dependencies.

$ go mod tidy

Open the main.go file in the terraform-provider-hashicups repository's root directory and replace the import declaration with the following.

main.go

import (
    "context"
    "flag"
    "log"

    "github.com/hashicorp/terraform-plugin-framework/providerserver"

    "terraform-provider-hashicups/internal/provider"
)

Create a docker_compose directory in the repository you cloned, which will contain the Docker configuration required to launch a local instance of HashiCups.

$ mkdir docker_compose

Create a docker_compose/conf.json file with the following.

docker_compose/conf.json

{
  "db_connection": "host=db port=5432 user=postgres password=password dbname=products sslmode=disable",
  "bind_address": "0.0.0.0:9090",
  "metrics_address": "localhost:9102"
}

Create a docker_compose/docker-compose.yml file with the following.

docker_compose/docker-compose.yml

version: '3.7'
services:
  api:
    image: "hashicorpdemoapp/product-api:v0.0.22"
    ports:
      - "19090:9090"
    volumes:
      - ./conf.json:/config/config.json
    environment:
      CONFIG_FILE: '/config/config.json'
    depends_on:
      - db
  db:
    image: "hashicorpdemoapp/product-api-db:v0.0.22"
    ports:
      - "15432:5432"
    environment:
      POSTGRES_DB: 'products'
      POSTGRES_USER: 'postgres'
      POSTGRES_PASSWORD: 'password'

If you are stuck at any point during this tutorial, refer to the 01-provider directory from the example repository.

Implement initial provider type

Providers use an implementation of the provider.Provider interface type as the starting point for all implementation details.

This interface requires the following:

A Metadata method to define the provider type name for inclusion in each data source and resource type name. For example, a resource type named "hashicups_order" would have a provider type name of "hashicups".
A Schema method to define the schema for provider-level configuration. Later in these tutorials, you will update this method to accept a HashiCups API token and endpoint.
A Configure method to configure shared clients for data source and resource implementations.
A DataSources method to define the provider's data sources.
A Resources method to define the provider's resources.

Go to the internal/provider directory in the repository you cloned, which will contain all the Go code for the provider except the provider server.

Open the internal/provider/provider.go file and replace the existing code with the following.

internal/provider/provider.go

package provider

import (
    "context"

    "github.com/hashicorp/terraform-plugin-framework/datasource"
    "github.com/hashicorp/terraform-plugin-framework/provider"
    "github.com/hashicorp/terraform-plugin-framework/provider/schema"
    "github.com/hashicorp/terraform-plugin-framework/resource"
)

// Ensure the implementation satisfies the expected interfaces.
var (
    _ provider.Provider = &hashicupsProvider{}
)

// New is a helper function to simplify provider server and testing implementation.
func New(version string) func() provider.Provider {
    return func() provider.Provider {
        return &hashicupsProvider{
            version: version,
        }
    }
}

// hashicupsProvider is the provider implementation.
type hashicupsProvider struct {
    // version is set to the provider version on release, "dev" when the
    // provider is built and ran locally, and "test" when running acceptance
    // testing.
    version string
}

// Metadata returns the provider type name.
func (p *hashicupsProvider) Metadata(_ context.Context, _ provider.MetadataRequest, resp *provider.MetadataResponse) {
    resp.TypeName = "hashicups"
    resp.Version = p.version
}

// Schema defines the provider-level schema for configuration data.
func (p *hashicupsProvider) Schema(_ context.Context, _ provider.SchemaRequest, resp *provider.SchemaResponse) {
    resp.Schema = schema.Schema{}
}

// Configure prepares a HashiCups API client for data sources and resources.
func (p *hashicupsProvider) Configure(ctx context.Context, req provider.ConfigureRequest, resp *provider.ConfigureResponse) {
}

// DataSources defines the data sources implemented in the provider.
func (p *hashicupsProvider) DataSources(_ context.Context) []func() datasource.DataSource {
    return nil
}

// Resources defines the resources implemented in the provider.
func (p *hashicupsProvider) Resources(_ context.Context) []func() resource.Resource {
    return nil
}

Implement the provider server

Terraform providers are server processes that Terraform interacts with to handle each data source and resource operation, such as creating a resource on a remote system. Later in these tutorials, you will connect those Terraform operations to a locally running HashiCups API.

Serving a provider follows these steps:

Starts a provider server process. By implementing the main function, which is the code execution starting point for Go language programs, a long-running server will listen for Terraform requests.

Framework provider servers also support optional functionality such as enabling support for debugging tools. You will not implement this functionality in these tutorials.

Open the main.go file in the terraform-provider-hashicups repository's root directory and replace the main function with the following.

main.go

func main() {
    var debug bool

    flag.BoolVar(&debug, "debug", false, "set to true to run the provider with support for debuggers like delve")
    flag.Parse()

    opts := providerserver.ServeOpts{
        // NOTE: This is not a typical Terraform Registry provider address,
        // such as registry.terraform.io/hashicorp/hashicups. This specific
        // provider address is used in these tutorials in conjunction with a
        // specific Terraform CLI configuration for manual development testing
        // of this provider.
        Address: "hashicorp.com/edu/hashicups",
        Debug:   debug,
    }

    err := providerserver.Serve(context.Background(), provider.New(version), opts)

    if err != nil {
        log.Fatal(err.Error())
    }
}

Verify the initial provider

With the Go dependencies ready, your provider code should compile and run. Verify that your development environment is working properly by executing the code directly. This will return an error message as this is not how Terraform normally starts provider servers, but the error indicates that Go was able to compile and run your provider server.

Manually run the provider.

$ go run main.go
This binary is a plugin. These are not meant to be executed directly.
Please execute the program that consumes these plugins, which will
load any plugins automatically
exit status 1

Prepare Terraform for local provider install

Terraform installs providers and verifies their versions and checksums when you run terraform init. Terraform will download your providers from either the provider registry or a local registry. However, while building your provider you will want to test Terraform configuration against a local development build of the provider. The development build will not have an associated version number or an official set of checksums listed in a provider registry.

Terraform allows you to use local provider builds by setting a dev_overrides block in a configuration file called .terraformrc. This block overrides all other configured installation methods.

Terraform searches for the .terraformrc file in your home directory and applies any configuration settings you set.

First, find the GOBIN path where Go installs your binaries. Your path may vary depending on how your Go environment variables are configured.

$ go env GOBIN
/Users/<Username>/go/bin

If the GOBIN go environment variable is not set, use the default path, /Users/<Username>/go/bin.

Create a new file called .terraformrc in your home directory (~), then add the dev_overrides block below. Change the <PATH> to the value returned from the go env GOBIN command above.

~/.terraformrc

provider_installation {

  dev_overrides {
      "hashicorp.com/edu/hashicups" = "<PATH>"
  }

  # For all other providers, install them directly from their origin provider
  # registries as normal. If you omit this, Terraform will _only_ use
  # the dev_overrides block, and so no other providers will be available.
  direct {}
}

You must create your CLI configuration file in your user's %APPDATA% directory. The physical location of this directory depends on your Windows version and system configuration.

Search for the user environment variables in PowerShell to find its location on your system.

$ $env:APPDATA

In another terminal, change into the directory returned from the command above.

$ cd /APPDATA_DIR

Find the GOBIN path where go installs your binaries. Your path may vary depending on how your Go environment variables are configured.

$ go env GOBIN
/Users/<Username>/go/bin

If the GOBIN go environment variable is not set, use the default path, /Users/<Username>/go/bin.

Create a new file in your APPDATA directory called terraform.rc and add the dev_overrides block below. Change the <PATH> to the value returned from the go env GOBIN command above.

terraform.rc

provider_installation {

  dev_overrides {
      "hashicorp.com/edu/hashicups" = "<PATH>"
      }

  # For all other providers, install them directly from their origin provider
  # registries as normal. If you omit this, Terraform will _only_ use
  # the dev_overrides block, and so no other providers will be available.
  direct {}
}

Locally install provider and verify with Terraform

Your Terraform CLI is now ready to use the locally installed provider in the GOBIN path. Use the go install command from the example repository's root directory to compile the provider into a binary and install it in your GOBIN path.

$ go install .

Create an examples/provider-install-verification directory, which will contain a terraform configuration to verify local provider installation, and navigate to it.

$ mkdir examples/provider-install-verification && cd "$_"

Create a main.tf file with the following.

examples/provider-install-verification/main.tf

terraform {
  required_providers {
    hashicups = {
      source = "hashicorp.com/edu/hashicups"
    }
  }
}

provider "hashicups" {}

data "hashicups_coffees" "example" {}

The main.tf Terraform configuration file in this directory uses a "hashicups_coffees" data source that the provider does not yet support. You will implement this data source in a future tutorial.

Running a Terraform plan will report the provider override, as well as an error about the missing data source. Even though there was an error, this verifies that Terraform was able to successfully start the locally installed provider and interact with it in your development environment.

Run a Terraform plan with the non-existent data source. Terraform will respond with the missing data source error.

$ terraform plan
╷
│ Warning: Provider development overrides are in effect
│
│ The following provider development overrides are set in the CLI
│ configuration:
│  - hashicorp.com/edu/hashicups in /Users/<Username>/go/bin
│
│ The behavior may therefore not match any released version of the provider and
│ applying changes may cause the state to become incompatible with published
│ releases.
╵
╷
│ Error: Invalid data source
│
│   on main.tf line 11, in data "hashicups_coffees" "example":
│   11: data "hashicups_coffees" "example" {}
│
│ The provider hashicorp.com/edu/hashicups does not support data source
│ "hashicups_coffees".
╵

Navigate to the terraform-provider-hashicups directory.

$ cd ../..

Next steps

Congratulations! You have started development of your own custom Terraform provider. Later tutorials will show you how to implement data source and resource functionality.

If you were stuck during this tutorial, checkout the 01-provider directory in the example repository to see the code implemented in this tutorial.

To learn more about the Terraform Plugin Framework, refer to the Terraform Plugin Framework documentation.
For a full capability comparison between the SDKv2 and the Plugin Framework, refer to the Which SDK Should I Use? documentation.
The example repository contains directories corresponding to each tutorial in this collection.
Submit any Terraform Plugin Framework bug reports or feature requests to the development team in the Terraform Plugin Framework Github repository.
Submit any Terraform Plugin Framework questions in the Terraform Plugin Framework Discuss forum.

Collection Overview

Custom Framework Providers

Configure client