Terraform — API Surface#

API types#

Terraform exposes functionality through three distinct surface areas:

CLI — the primary human-facing interface (terraform plan, terraform apply, etc.)
gRPC Plugin Protocol — the provider/provisioner contract (tfplugin5.proto / tfplugin6.proto)
gRPC RPC API (rpcapi) — a machine-facing gRPC interface for automation and HCP Terraform

There is no HTTP/REST API and no exported Go library API. Every internal package is internal/, making external programmatic access impossible without the RPC API.

CLI#

Framework#

github.com/hashicorp/cli (Mitchell Hashimoto’s CLI library, not cobra). Commands are registered as a map[string]cli.CommandFactory at startup in commands.go. Flags are parsed via stdlib flag through a helper extendedFlagSet(name, ...) that assembles reusable flag groups.

Command structure#

Commands are organized in three tiers: primary workflow (advertised prominently), plumbing (useful but secondary), and hidden (legacy aliases or internal plumbing).

Primary workflow commands#

Command	Purpose
`init`	Initialize a working directory; install providers and modules
`validate`	Validate configuration files syntactically and semantically
`plan`	Create an execution plan, show what changes will be made
`apply`	Execute the plan, making infrastructure changes
`destroy`	Destroy all managed infrastructure (alias: `apply -destroy`)

Infrastructure management#

Command	Purpose
`fmt`	Reformat `.tf` files to canonical style
`get`	Download module dependencies
`import`	Import existing infrastructure into state
`refresh`	Update state file against real infrastructure
`output`	Read and display output values
`show`	Display human-readable state or plan file
`graph`	Output DOT-format resource dependency graph
`console`	Interactive REPL for expression evaluation
`query`	Query infrastructure state

Provider management#

Command	Purpose
`providers`	Show provider requirements and selections
`providers lock`	Update `.terraform.lock.hcl` with provider checksums
`providers mirror`	Mirror providers to a local filesystem directory
`providers schema`	Print provider schema in JSON
`login`	Obtain and store credentials for a Terraform host
`logout`	Remove stored credentials for a Terraform host
`metadata functions`	Show provider function metadata

State management (plumbing)#

Command	Purpose
`state list`	List resources in state
`state identities`	List resource identity attributes
`state show`	Show attributes of a single resource
`state mv`	Move an item in state (rename)
`state rm`	Remove a resource from state
`state pull`	Output raw state to stdout
`state push`	Push a local state file to remote
`state replace-provider`	Replace provider in state
`force-unlock`	Release a stuck state lock
`taint` / `untaint`	Mark resource for re-creation (deprecated)

Workspace management#

Command	Purpose
`workspace list`	List workspaces
`workspace select`	Switch to a workspace
`workspace show`	Show current workspace name
`workspace new`	Create a new workspace
`workspace delete`	Delete a workspace

Stacks (multi-configuration orchestration)#

Command	Purpose
`stacks`	Stacks subcommand stub (delegates to rpcapi internally)

Hidden/legacy commands#

Command	Notes
`rpcapi`	Machine-facing gRPC entry point; hidden from help
`env` / `env list/select/new/delete`	Legacy aliases for `workspace` commands
`push`	Removed feature stub
`internal-plugin`	Internal plugin server launcher

Experimental (gated by `ExperimentsAllowed`)#

Command	Purpose
`cloud`	HCP Terraform integration commands
`test cleanup`	Clean up resources after `terraform test`

Flag patterns#

Flags are parsed using stdlib flag via a per-command extendedFlagSet. Arguments are decomposed into typed reusable structs in internal/command/arguments/:

arguments.View — global: -no-color, -compact-warnings, -json
arguments.State — -state, -state-out, -backup
arguments.Operation — -auto-approve, -parallelism, -refresh, -refresh-only, -replace, -target
arguments.Vars — -var, -var-file
Per-command extras: e.g. Plan adds -out, -detailed-exitcode, -generate-config-out

Global flag: -chdir=<dir> is a pre-parse special flag handled before command dispatch in main.go, changing the working directory before running any command.

Persistent env var injection: TF_CLI_ARGS and TF_CLI_ARGS_<command> are read at startup and prepended to os.Args before parsing, allowing automation to inject flags globally.

Output format#

Every command supports two output modes:

Human (default): colored, TTY-aware, with ANSI formatting. Width adapts to terminal columns.
JSON (-json): newline-delimited JSON messages, designed for machine parsing. Structured via internal/command/json*/ renderers.

The views/ sub-package abstracts the choice: each command gets a views.Plan, views.Apply, etc. type that dispatches to either the human or JSON renderer.

gRPC Plugin Protocol (Provider / Provisioner API)#

This is the contract between Terraform Core and provider binaries. It is the most public API surface in Terraform — thousands of provider binaries implement it.

Proto files#

internal/tfplugin5/tfplugin5.proto — Protocol 5 (SDK v2 providers)
internal/tfplugin6/tfplugin6.proto — Protocol 6 (SDK v2 with new features)
docs/plugin-protocol/tfplugin5.proto / tfplugin6.proto — canonical reference copies

Both protocols are maintained simultaneously. The grpcwrap package adapts providers.Interface Go implementations to serve either protocol. The plugin and plugin6 packages provide the gRPC client stubs (one per protocol version).

Protocol 5 — `service Provider`#

RPC	Purpose
`GetMetadata`	Provider capabilities/metadata
`GetSchema`	Full provider, resource, data-source schemas
`PrepareProviderConfig`	Validate and normalize provider config
`ValidateResourceTypeConfig`	Validate resource config
`ValidateDataSourceConfig`	Validate data source config
`UpgradeResourceState`	Upgrade persisted state from older schema version
`GetResourceIdentitySchemas`	Identity attribute schemas
`UpgradeResourceIdentity`	Upgrade stored identity
`Configure`	Pass configured provider credentials/settings
`ReadResource`	Refresh resource state against real infrastructure
`PlanResourceChange`	Compute diff for a resource
`ApplyResourceChange`	Apply resource CRUD operations
`ImportResourceState`	Import existing resource into state
`MoveResourceState`	Move resource to another provider instance
`ReadDataSource`	Read a data source
`GenerateResourceConfig`	Generate HCL config for imported resource
`ValidateEphemeralResourceConfig`	Validate ephemeral resource config
`OpenEphemeralResource`	Open an ephemeral resource lifetime
`RenewEphemeralResource`	Renew ephemeral resource lease
`CloseEphemeralResource`	Close ephemeral resource lifetime
`ListResource`	Stream resource list results
`ValidateListResourceConfig`	Validate list resource config
`GetFunctions`	Get provider-defined functions
`CallFunction`	Call a provider-defined function
`PlanAction`	Plan a provider-defined action
`InvokeAction`	Invoke a provider-defined action (streaming)
`ValidateActionConfig`	Validate action config
`Stop`	Graceful shutdown signal

Protocol 6 — `service Provider`#

Identical set of RPCs as Protocol 5 with the same semantics. Protocol 6 adds native support for newer Terraform features (e.g., ephemeral resources, list resources, actions, state stores) while Protocol 5 carries polyfills for backward compatibility.

State Store RPCs (Protocol 6 extension):

RPC	Purpose
`ValidateStateStoreConfig`	Validate state store backend config
`ConfigureStateStore`	Configure the state store
`ReadStateBytes`	Stream state bytes from provider-managed store
`WriteStateBytes`	Stream state bytes to provider-managed store
`LockState`	Acquire state lock
`UnlockState`	Release state lock
`GetStates`	List all state entries
`DeleteState`	Delete a state entry

`service Provisioner` (Protocol 5, deprecated)#

RPC	Purpose
`GetSchema`	Provisioner config schema
`ValidateProvisionerConfig`	Validate config
`ProvisionResource`	Run provisioner, streaming log output
`Stop`	Graceful stop

Transport#

Providers are launched as subprocesses by hashicorp/go-plugin. The plugin protocol uses gRPC over a loopback TCP listener (or Unix socket), with the address negotiated over stdin/stdout using a magic cookie handshake. The subprocess is verified by matching MAGIC_COOKIE environment variables.

gRPC RPC API (`rpcapi`)#

The rpcapi subsystem exposes Terraform Core as a go-plugin gRPC server, intended for consumption by HCP Terraform and other automation. It is launched via the hidden rpcapi CLI command.

Entry point#

rpcapi.ServePlugin(ctx, opts) — validates the go-plugin magic cookie (TERRAFORM_RPCAPI_COOKIE), then serves protocol version 1 with a tfcore plugin set.

Proto files and services#

All live under internal/rpcapi/terraform1/:

`setup/setup.proto` — `service Setup`#

Capability negotiation between client and server.

RPC	Purpose
`Handshake`	Version negotiation, capability exchange
`Stop`	Graceful server shutdown

`dependencies/dependencies.proto` — `service Dependencies`#

Provider installation and lock file management.

RPC	Purpose
`OpenSourceBundle` / `CloseSourceBundle`	Open a source bundle (config + modules)
`OpenDependencyLockFile` / `CreateDependencyLocks` / `CloseDependencyLocks`	Manage `.terraform.lock.hcl`
`GetLockedProviderDependencies`	Query locked provider versions
`BuildProviderPluginCache`	Install providers (streaming progress)
`OpenProviderPluginCache` / `CloseProviderPluginCache`	Manage provider cache handle
`GetCachedProviders`	List cached providers
`GetBuiltInProviders`	List built-in providers
`GetProviderSchema`	Retrieve provider schema

`packages/packages.proto` — `service Packages`#

Provider and module package registry operations (no session state required).

RPC	Purpose
`ProviderPackageVersions`	List available versions for a provider
`FetchProviderPackage`	Download a provider package
`ModulePackageVersions`	List available module versions
`ModulePackageSourceAddr`	Resolve module source address
`FetchModulePackage`	Download a module package

`stacks/stacks.proto` — `service Stacks`#

The Stacks execution model for multi-configuration orchestration.

RPC	Purpose
`OpenStackConfiguration` / `CloseStackConfiguration`	Open a `.tfstack.hcl` config
`ValidateStackConfiguration`	Validate a stack configuration
`FindStackConfigurationComponents`	Enumerate components in a stack
`OpenState` / `CloseState`	Open a Stacks state object (streamed)
`PlanStackChanges`	Compute a stacks plan (streaming events)
`OpenPlan` / `ClosePlan`	Handle plan objects
`ApplyStackChanges`	Apply a stacks plan (streaming events)
`OpenStackInspector` / `InspectExpressionResult`	Interactive expression evaluation
`OpenTerraformState` / `CloseTerraformState`	Wrap classic Terraform state for migration
`MigrateTerraformState`	Migrate classic state into Stacks
`ListResourceIdentities`	List identities of managed resources

OTel interceptors#

Both unary and streaming gRPC calls are instrumented with OpenTelemetry via otelgrpc.UnaryServerInterceptor() / otelgrpc.StreamServerInterceptor() (set up in server.go).

Plugin / Extension system#

Provider plugins (`hashicorp/go-plugin` + gRPC)#

Mechanism: Out-of-process subprocess, gRPC over go-plugin
Extension points:
- Implement service Provider in either tfplugin5.proto or tfplugin6.proto
- Provider binaries discovered via registry, filesystem mirror, or dev override
- providers.Interface is the Go-side abstraction; 30+ methods
Registration: ContextOpts.Providers map[addrs.Provider]providers.Factory; built-in providers (e.g., terraform, registry.terraform.io/hashicorp/null) are registered the same way but never launched as subprocesses

Stacks / Cloud plugins (`stacksproto1.proto`, `cloudproto1.proto`)#

Mechanism: CommandService.Execute — a single streaming RPC that proxies sub-commands from Terraform Core to the cloud/stacks plugin process
Extension points: internal/stacksplugin and internal/cloudplugin — separate plugin processes for HCP Terraform’s Stacks and Cloud features

Credentials helpers#

Mechanism: ~/.terraformrc → credentials_helper block → external binary that speaks the credential helper protocol (JSON over stdin/stdout)
Discovered via pluginDiscovery.FindPlugins("credentials", globalPluginDirs)

Backend plugins (state storage)#

Not a runtime plugin system — the nine remote state backends (S3, GCS, Azure, Consul, k8s, PG, OCI, COS, OSS) are separate Go modules in the same repository with their own go.mod files, wired via replace directives
Each implements backend.Backend and statemgr.Full

Summary of key design choices#

No exported library API. internal/ everywhere is intentional. External tools must use the RPC API or the plugin protocol.
Two machine-facing surfaces with different audiences. The plugin protocol (tfplugin5/6) is for provider authors; the rpcapi gRPC is for orchestration tooling (HCP Terraform). They are entirely separate wire formats.
Dual protocol maintenance. Protocol 5 and 6 coexist because the provider ecosystem cannot be upgraded atomically. New features are Protocol-6-only; Protocol 5 receives minimal maintenance.
CLI output duality. Every command outputs either human-readable or structured JSON (-json). The views/ abstraction makes this systematic rather than ad hoc.
Flag composition over inheritance. CLI flags are assembled from reusable typed structs (State, Operation, Vars) rather than a deep flag-inheritance hierarchy. This keeps each command’s accepted flags explicit and documentable.