Migrating a Terraform Provider from dep to Go Modules

Migrating a Terraform Provider from dep to Go Modules

Recently, I've had to start maintaining a fork of terraform-provider-k8s, as part of our workflow at work has been applying raw Kubernetes manifests via Terraform. The official Terraform Kubernetes provider doesn't support applying arbitrary YAML manifests. Our teams work with minikube locally and directly with kubectl in dev. Being able to use the standard form in later deployment environments gets us the best of both worlds.

We've been running this provider for a while now and it's still using dep for dependency management. With Terraform 0.12 recently released and Go modules now on by default, it's time to modernise the repo. This post covers the migration from dep to Go 1.11 modules and Terraform 0.12 update.

Go Modules?

The Go developer made Modules available in Go 1.11 back in August last year, with the setting becoming default on via GO111MODULE in Go 1.12 in February. While dep had its place previously, now the language has official tooling it is time to migrate.

For us there are a few practical motivations:

  • Terraform 0.12 compatibility - Terraform 0.12 landed in May and brings significant changes to the plugin SDK. Since 0.11.12 they have been using Go modules.
  • Simpler CI - With modules, go build and go test is all we need. Who doesn't love to drop a tooling dependency.

Migrating

1. Initialise the module

First, we initialise the Go module.

$ go mod init github.com/fiveai/terraform-provider-k8s

This generates a go.mod and go.sum file. The go.mod file is refreshingly simple compared to Gopkg.toml:

2. Tidy up

$ go mod tidy

This prunes unneeded dependencies and adds any that are missing. Good practice to run this after the initial conversion to make sure everything is clean.

3. Verify and test

$ go build ./...
$ go test ./...

If everything compiles and the tests pass, you're in good shape. For us, this just worked, no need to change or update anything.

4. Remove dep files

Once you're happy, remove the old dep files:

$ rm Gopkg.toml Gopkg.lock
$ rm -rf vendor/

Updating to Terraform 0.12

With modules in place, updating to Terraform 0.12 was just a matter of updating the dependency versions in go.mod. Terraform 0.12 brings some breaking changes to the plugin SDK but for our provider the impact is minimal since we mostly shell out to kubectl rather than using the SDK's resource helpers extensively.

Release Tooling

While I was making changes, I decided I might as well add cross-compiling to other platforms. The Makefile builds for Linux, macOS and Windows across amd64 which covers our team's needs, every dev likes their own environment. It isn't the year of the Linux Desktop, yet.

BINARY=terraform-provider-k8s

build:
	go build -o $(BINARY)

release:
	GOOS=linux GOARCH=amd64 go build -o $(BINARY)_linux_amd64
	GOOS=darwin GOARCH=amd64 go build -o $(BINARY)_darwin_amd64
	GOOS=windows GOARCH=amd64 go build -o $(BINARY)_windows_amd64.exe

Summary

The migration from dep to Go modules was smoother than expected (how many waves of Python package management has there been now?) Combined with Terraform 0.12 and some release tooling, we've brought the provider up to date with where the Go and Terraform ecosystems are heading.

As always, I appreciate any feedback or if you want to reach out, I'm @neuralsandwich on twitter and most other places.