Terraform — Structure#

Layout pattern#

Custom: Root-package main + all-internal monorepo with embedded sub-modules

Terraform departs from the canonical Standard Go Layout in two notable ways. First, there is no cmd/ directory — the main binary is built directly from the repository root (package main in main.go, commands.go, working_dir.go, etc.). Second, the project exposes no public packages: everything outside the root main package lives under internal/, making the entire codebase private to the module. A third structural quirk — the remote-state backends are modelled as separate go.mod modules nested inside internal/backend/remote-state/ and wired back in via replace directives — gives the repo characteristics of a monorepo even though it publishes only one binary.

Directory map#

repositories/terraform/
├── main.go                        # Entry point: realMain(), CLI bootstrap
├── commands.go                    # initCommands() — registers all CLI commands
├── working_dir.go                 # WorkingDir helper used by commands
├── provider_source.go             # Provider source / installation config
├── telemetry.go                   # OpenTelemetry initialization
├── checkpoint.go                  # HashiCorp checkpoint (version check) runner
├── experiments.go                 # Experiment feature-flag logic
├── version.go                     # Version constants
├── go.mod / go.sum                # Single module with replace directives
├── Makefile                       # generate, protobuf, fmtcheck, vetcheck, staticcheck
├── Dockerfile                     # Dev-only Docker build (not official release)
├── build.Dockerfile               # Alternative builder image
├── BUILDING.md                    # Build instructions (ldflags for dev/release/experiments)
├── docs/                          # Plugin protocol docs, debug docs, images
├── scripts/                       # Shell scripts: gofmtcheck, staticcheck, build.sh, etc.
├── testing/                       # Equivalence test fixtures for plan/apply golden files
├── tools/                         # Developer tools (not shipped in binary)
│   ├── loggraphdiff/              # CLI tool to diff graph walk logs
│   ├── protobuf-compile/          # Runs protoc for tfplugin5/6 and rpcapi protos
│   └── terraform-bundle/          # CLI tool to bundle providers (deprecated)
├── version/                       # version.go: version string + InterestingDependencies()
└── internal/                      # ALL application code — no public packages
    ├── terraform/                 # [154 src files] Core: graph engine, Context, plan/apply walk
    ├── command/                   # [69 src files] CLI subcommand implementations
    │   ├── arguments/             # Typed argument structs for each command
    │   ├── cliconfig/             # ~/.terraformrc and TERRAFORM_CONFIG_FILE parsing
    │   ├── clistate/              # State locking helpers for commands
    │   ├── format/                # Human-readable output formatters (pre-JSON era)
    │   ├── jsonconfig/jsonformat/jsonplan/jsonstate/jsonprovider/jsonchecks/jsonfunction/
    │   │                          # Machine-readable JSON output renderers
    │   ├── views/                 # Output abstraction layer (human vs JSON modes)
    │   ├── junit/                 # JUnit XML output for `terraform test`
    │   ├── webbrowser/            # Cross-platform browser launcher (for `login`)
    │   ├── workdir/               # Working directory abstraction (.terraform/ dir)
    │   └── e2etest/               # End-to-end test helpers (subprocess-based)
    ├── configs/                   # [41 src files] HCL config parsing: Module, Resource, Provider, etc.
    │   ├── configload/            # Loads module trees from disk/registry
    │   ├── configschema/          # Schema types bridging HCL and provider schemas
    │   ├── hcl2shim/              # Shims between HCL v2 and legacy SDK schema types
    │   └── configtesting/         # Test helpers for config loading
    ├── addrs/                     # [46 src files] Address types for every referenceable object
    │   │                          # (ResourceInstance, Provider, Module, InputVariable, etc.)
    ├── lang/                      # Built-in functions, expression scope/evaluation
    │   └── funcs/                 # Individual function implementations (ceil, regex, yamldecode, …)
    ├── dag/                       # Directed acyclic graph: Walk, TopologicalSort, Tarjan SCC, dot output
    ├── states/                    # State data model (statefile v4 JSON, StateMgr interfaces)
    │   ├── statemgr/              # State manager implementations (local file, generic)
    │   └── statefile/             # statefile.v4 JSON serialization
    ├── plans/                     # Plan data structures (Changes, ChangesSrc, DynamicValue)
    ├── backend/                   # State backend framework + implementations
    │   ├── init/                  # Backend registry (maps names → constructors)
    │   ├── local/                 # Local backend (default: run graph walk in-process)
    │   ├── remote/                # Remote backend (HCP Terraform API streaming)
    │   ├── backendbase/           # Shared backend helpers
    │   ├── backendrun/            # RunningBackend interface for plan/apply
    │   ├── pluggable/             # Pluggable backend wrapper
    │   └── remote-state/          # [9 sub-modules with their own go.mod]
    │       ├── s3/                # AWS S3 + DynamoDB locking
    │       ├── gcs/               # Google Cloud Storage
    │       ├── azure/             # Azure Blob Storage
    │       ├── consul/            # HashiCorp Consul
    │       ├── kubernetes/        # Kubernetes Secrets
    │       ├── pg/                # PostgreSQL
    │       ├── oci/               # Oracle Cloud Infrastructure
    │       ├── cos/               # Tencent Cloud Object Storage
    │       ├── oss/               # Alibaba Cloud Object Storage
    │       ├── http/              # Generic HTTP (pure Go, no sub-module)
    │       └── inmem/             # In-memory (testing only, no sub-module)
    ├── providers/                 # Provider interface definitions + schema types
    ├── plugin/                    # Plugin5 client (grpc, wraps go-plugin, Protocol 5)
    │   └── discovery/             # Plugin discovery from filesystem
    ├── plugin6/                   # Plugin6 client (Protocol 6)
    ├── pluginshared/              # Shared types between plugin5 and plugin6
    ├── tfplugin5/                 # Protobuf-generated stubs: tfplugin5.proto (Protocol 5)
    ├── tfplugin6/                 # Protobuf-generated stubs: tfplugin6.proto (Protocol 6)
    ├── grpcwrap/                  # Adapters: wraps providers.Interface → gRPC server impl
    ├── rpcapi/                    # [18 src files] Machine-facing RPC API (bypasses CLI layer)
    │   ├── terraform1/            # terraform1.proto stubs (Stacks RPC service)
    │   └── dynrpcserver/          # Code-generated dynamic RPC server dispatcher
    ├── stacks/                    # Stacks feature (newer orchestration layer)
    │   ├── stackaddrs/            # Address types specific to Stacks
    │   ├── stackconfig/           # HCL parsing for .tfstack.hcl files
    │   ├── stackplan/             # Stacks plan data structures
    │   ├── stackstate/            # Stacks state data structures
    │   ├── stackruntime/          # Stacks execution engine
    │   │   └── internal/stackeval # [52 src files] Stacks evaluator (largest sub-package)
    │   ├── stackmigrate/          # Migration helpers from classic modules to Stacks
    │   └── tfstackdata1/          # Protobuf-generated stubs for Stacks state/plan wire format
    ├── stacksplugin/              # Stacks-specific plugin protocol
    │   └── stacksproto1/          # stacksproto1.proto stubs
    ├── cloudplugin/               # HCP Terraform plugin protocol (cloud-specific extension)
    ├── cloud/                     # HCP Terraform / Terraform Cloud integration
    ├── registry/                  # Terraform Registry client (module + provider download)
    ├── getproviders/              # Provider installation: FS layout, lock file, mirror logic
    ├── getmodules/                # Module installation (git, registry, local)
    ├── initwd/                    # `terraform init` working directory initialization
    ├── providercache/             # Provider cache directory management
    ├── depsfile/                  # .terraform.lock.hcl parser/writer
    ├── tfdiags/                   # Diagnostics framework (wraps HCL diags + custom types)
    ├── checks/                    # Check block and assertion tracking
    ├── namedvals/                 # Named values scope (vars, locals, outputs during eval)
    ├── instances/                 # Instance key tracking (for_each expansion)
    ├── instances/                 # for_each / count instance key types
    ├── refactoring/               # moved/removed block processing for state migration
    ├── genconfig/                 # Generate HCL config from imported resources
    ├── promising/                 # Promise/future abstraction for async evaluation
    ├── moduletest/                # `terraform test` framework internals
    │   └── graph/                 # [18 src files] Test graph walk engine
    ├── legacy/                    # Backward-compat shims for old SDK/helper/schema
    ├── terminal/                  # Terminal stream abstraction (color, width detection)
    ├── logging/                   # Log-level routing, plugin panic capture
    ├── experiments/               # Experiment (feature flag) registry
    ├── deprecation/               # Deprecation warning framework
    ├── communicator/              # SSH/WinRM communicators (for provisioners)
    ├── provisioners/              # Provisioner interface
    ├── provisioner-local-exec/    # local-exec provisioner (built-in, shipped as subprocess)
    ├── provider-simple/           # Test-only Protocol 5 provider
    ├── provider-simple-v6/        # Test-only Protocol 6 provider
    ├── provider-terraform/        # `terraform` provider (manages workspaces/state)
    ├── repl/                      # REPL evaluator for `terraform console`
    ├── schemarepo/                # Schema repository (in-memory cache of provider schemas)
    ├── resources/                 # Resource identity types
    ├── actions/                   # Action types (new resource lifecycle feature)
    ├── collections/               # Generic collection types (Set, Map with cty key)
    ├── ipaddr/                    # IP address utility (CIDR helpers)
    ├── copy/                      # File copy utilities
    ├── helper/                    # Miscellaneous helpers
    ├── modsdir/                   # Module manifest (.terraform/modules/modules.json)
    ├── moduledeps/                # Module dependency graph (for init)
    ├── moduleref/                 # Module reference tracking
    ├── releaseauth/               # Release binary authentication (signature verification)
    ├── replacefile/               # Atomic file replacement helper
    ├── httpclient/                # Shared HTTP client with Terraform User-Agent
    ├── didyoumean/                # Levenshtein "did you mean?" suggestion helper
    └── e2e/                       # E2E test subprocess harness

Entry points#

Binary	Source	Purpose
`terraform`	`./main.go`	Primary CLI binary — all subcommands
(test provider v5)	`internal/provider-simple/main/main.go`	Protocol 5 test provider used in acceptance tests
(test provider v6)	`internal/provider-simple-v6/main/main.go`	Protocol 6 test provider used in acceptance tests
(terraform provider)	`internal/provider-terraform/main/main.go`	The `terraform` provider (manages TF state/workspaces)
(local-exec provisioner)	`internal/provisioner-local-exec/main/main.go`	local-exec built-in provisioner subprocess
(rpcapi generator)	`internal/rpcapi/dynrpcserver/generator/main.go`	Code generator for dynamic RPC server dispatcher
(stackeval generator)	`internal/stacks/stackruntime/internal/stackeval/main.go`	Code generator for Stacks evaluator boilerplate

The only user-facing binary is the root terraform. The others are either shipped as subprocesses (providers, provisioner) or are build-time code generators.

Package organization#

Internal packages (selected significant ones)#

Package	Src files	Purpose
`internal/terraform`	154	Core: `Context`, graph builder, walk engine, plan/apply/import/eval
`internal/command`	69 (root)	All CLI subcommand structs + `Meta` shared base
`internal/stacks/stackruntime/internal/stackeval`	52	Stacks concurrent evaluator
`internal/addrs`	46	~40 address types for every referenceable HCL object
`internal/configs`	41	HCL module/resource/provider/variable parsing
`internal/command/arguments`	33	Typed argument structs per command
`internal/legacy/helper/schema`	26	Legacy SDK schema (backward compat)
`internal/cloud`	24	HCP Terraform cloud integration
`internal/tfdiags`	20	Diagnostics framework (source-located errors)
`internal/getproviders`	17	Provider installation protocol and filesystem layout
`internal/plans`	16	Plan data structures (Changes, ChangesSrc)
`internal/states`	13	State model (Module, Resource, Instance)
`internal/dag`	9	Directed acyclic graph (Walk, SCC, dot output)
`internal/promising`	4	Promise/future for async Stacks evaluation

Public packages (pkg/)#

None. Terraform exports zero packages. Every package outside the root main is under internal/. This is a deliberate design: Terraform core is not intended to be used as a library. Third-party tooling integrates through the provider plugin protocol (gRPC/go-plugin) or through the new rpcapi interface, never by importing packages.

Layering#

The dependency graph flows in one direction:

main (root) → command → terraform (core) → configs / addrs / dag
                      → backend → states / plans
                      → providers / plugin / plugin6
                      → tfdiags ← (used by almost everything)

There is a secondary “Stacks” vertical that mirrors this but targets rpcapi rather than the CLI:

rpcapi → stacks/stackruntime → stacks/stackconfig / stackaddrs / stackplan / stackstate
       → stacksplugin (separate gRPC protocol)

The internal/dag package is the foundation — it has no internal dependencies and is imported by both internal/terraform and the Stacks graph walker in internal/moduletest/graph.

Multi-module remote-state backends#

Nine remote-state backends are separate Go modules, each with its own go.mod, and are connected to the root module via replace directives:

github.com/hashicorp/terraform/internal/backend/remote-state/s3        → ./internal/backend/remote-state/s3
github.com/hashicorp/terraform/internal/backend/remote-state/gcs       → ./internal/backend/remote-state/gcs
github.com/hashicorp/terraform/internal/backend/remote-state/azure     → ./internal/backend/remote-state/azure
github.com/hashicorp/terraform/internal/backend/remote-state/consul    → ./internal/backend/remote-state/consul
github.com/hashicorp/terraform/internal/backend/remote-state/kubernetes → ./internal/backend/remote-state/kubernetes
github.com/hashicorp/terraform/internal/backend/remote-state/pg        → ./internal/backend/remote-state/pg
github.com/hashicorp/terraform/internal/backend/remote-state/oci       → ./internal/backend/remote-state/oci
github.com/hashicorp/terraform/internal/backend/remote-state/cos       → ./internal/backend/remote-state/cos
github.com/hashicorp/terraform/internal/backend/remote-state/oss       → ./internal/backend/remote-state/oss

http and inmem backends have no sub-module (no cloud SDK dependencies) and live as plain packages within the root module.

Build system#

Build tool: go build . (root) for the binary; Makefile for developer workflow targets only — no custom build system.
Key Makefile targets:
- generate — runs go generate ./... (string-er, mock generation)
- protobuf — runs go run ./tools/protobuf-compile . to regenerate tfplugin5, tfplugin6, rpcapi/terraform1, and stacksplugin/stacksproto1 protobuf stubs using pinned versions of protoc + Go plugins
- fmtcheck / importscheck — gofmt and goimports lint
- vetcheck — go vet ./...
- staticcheck — runs staticcheck via scripts/staticcheck.sh
- exhaustive — checks exhaustiveness of switch statements over enums
- copyright / copyrightfix — enforces BUSL-1.1 copyright headers
- syncdeps — syncs dependencies across all sub-modules
Docker: Yes — Dockerfile and build.Dockerfile. These are for development and CI only; official release binaries are built via a closed-source release pipeline using scripts/build.sh.
Release: ldflags control version.dev (dev suffix) and main.experimentsAllowed (experimental feature gates). No goreleaser.

Notable structural decisions#

Root package as main (no cmd/ directory). For a project of this scale, placing the main package at the repository root is unusual. It works because Terraform is a single-binary product with no library consumers — the root is both the module root and the binary entry point. This simplifies go build . invocations but means the bootstrap logic (main.go, commands.go, working_dir.go, provider_source.go) is interleaved with the module metadata at the top level.
Zero exported packages — all-internal/. The internal/ constraint is applied to 100% of application code. This is an explicit architectural stance: Terraform core is a black box that external tooling reaches through gRPC protocols, not Go imports. The new rpcapi interface reinforces this — it is the intentional machine-facing entry point, explicitly documented to bypass the CLI layer for automation (see commands.go comments on the "rpcapi" command).
Multi-module monorepo for remote-state backends. Each remote-state backend carrying cloud-SDK dependencies (AWS SDK v2, GCP Go SDK, Azure SDK, etc.) has its own go.mod. The overview analysis noted this is self-described as “technical debt maintained for code-ownership clarity.” The benefit is that upgrading the AWS SDK for S3 doesn’t force rebuilding Terraform with the GCP SDK, and each backend can be reviewed and tested independently. The cost is complexity in go.mod and dependency synchronization (make syncdeps).
Dual plugin protocols maintained simultaneously. internal/tfplugin5/ and internal/tfplugin6/ are separate protobuf-generated packages, each with their own .proto source, bridged by adapters in internal/grpcwrap/. This supports providers compiled against either protocol without requiring all providers to upgrade simultaneously — a compatibility engineering challenge that has its own structural footprint.
Stacks as a parallel architecture track. The internal/stacks/ subtree (with stackruntime, stackconfig, stackaddrs, stackstate, stackplan, and tfstackdata1) mirrors the classic Terraform architecture but replaces the CLI-centric flow with an RPC-driven model (rpcapi). It has its own protobuf protocol (stacksplugin/stacksproto1), its own address types, and its own evaluator. This is not an evolution of the existing code — it is a parallel implementation intended for HCP Terraform’s orchestration layer, co-existing in the same repository while sharing dag, addrs primitives, and the provider plugin protocol.
internal/promising/ — custom async primitive. The promising package provides a promise/future abstraction (with deadlock detection, documented with an embedded PNG diagram deadlock-free-promises.png) specifically for the concurrent evaluation needs of the Stacks runtime. This is a rare case of a project implementing its own concurrency primitive rather than composing from sync or errgroup.
Generated code is checked in. The protobuf stubs (*.pb.go, *_grpc.pb.go) and string-er generated files (*_string.go) are committed to the repository. The make protobuf and make generate targets regenerate them, but the generated files are the source of truth for CI. This is standard HashiCorp practice and avoids requiring protoc as a build-time dependency for most contributors.