Terraform — Patterns#

Concurrency patterns#

Parallel DAG Walk (V×2 Goroutines)#

  • Usage: The core execution model for every plan/apply/destroy/validate operation
  • Example: internal/dag/walk.go:309 — go w.waitDeps(v, deps, doneCh, cancelCh) and go w.walkVertex(v, w.vertexMap[v]) — one goroutine per vertex for execution, one per vertex as a dependency waiter
  • Assessment: Highly effective for Terraform’s use case. The Walker supports mid-walk graph mutations (vertices/edges can be added while walking), which powers dynamic expansion nodes (count/for_each). sync.WaitGroup + sync.Mutex protect the shared vertexMap. Diagnostics are accumulated under diagsLock so all parallel errors are surfaced. The comment explicitly notes V×2 goroutines, treating that as a known, acceptable cost.

Done-Channel + Background Goroutine#

  • Usage: Backend operation progress monitoring in local.Local and remote.Remote — 4 occurrences in backend_plan.go, backend_apply.go, backend_refresh.go, backend.go
  • Example: internal/backend/local/backend_plan.go:115 — doneCh := make(chan struct{}) + go func() { defer close(doneCh); ... }() — the background goroutine closes the channel on completion; the caller selects on <-doneCh vs. <-ctx.Done()
  • Assessment: Classic done-channel idiom. Predates widespread context adoption in this area of the codebase. Functions correctly and is easy to reason about. New code in the same package prefers ctx.Done().

Context Cancellation#

  • Usage: 102 uses of ctx.Done(), context.WithCancel, context.WithTimeout, context.WithDeadline across the codebase; context.Context appears 1756 times total
  • Example: internal/dag/walk.go — the Walker accepts a context and its cancelCh is derived from it; graph nodes receive EvalContext which carries the Go context
  • Assessment: Context is threaded deeply through the entire stack — from CLI command through backend, terraform.Context, graph walker, down to each graph node. The integration is thorough and idiomatic. The sheer count (1756) reflects the depth of the call stack.

Graceful Shutdown via Signal + Hook#

  • Usage: commands.go:488-492 — SIGINT and SIGTERM are caught; the stopHook (an implementation of terraform.Hook) uses an atomic.Uint32 to signal the parallel walk to halt
  • Example: internal/terraform/hook_stop.go:19-31 — stopHook.hook() checks atomic.LoadUint32(&h.stop); if nonzero, returns HookActionHalt causing the walker to stop cleanly after the in-flight resource operation completes
  • Assessment: Elegant design: instead of directly canceling a context (which would leave state inconsistent), Terraform signals a “soft stop” via the Hook observer mechanism, allowing the current resource operation to complete, then halts. A second SIGINT triggers a hard kill. This gives a much safer shutdown than raw context cancellation.

Promise-Based Async (Stacks Runtime)#

  • Usage: internal/promising/ — used exclusively by the Stacks runtime (internal/stacks/stackeval/) as an alternative to the goroutine-walker model
  • Example: internal/promising/promise.go:84 — func NewPromise[T any](ctx context.Context, name string) (PromiseResolver[T], PromiseGet[T]) — generic future type; Once[T any] for idempotent lazy resolution
  • Assessment: A genuinely novel pattern in Go infrastructure tools. The promising package implements deadlock detection — if a task waits on a promise it is itself responsible for resolving, the runtime detects the cycle and panics with a useful diagnostic. Generic Promise[T], Once[T], PromiseResolver[T] are all Go 1.18+ generics.

Error handling#

  • Style: Mixed — tfdiags.Diagnostics is the primary error propagation mechanism for the internal stack; bare error is used at stdlib boundaries and lower-level utilities
  • Error types defined:
    • tfdiags.Diagnostics — slice of Diagnostic (interface); primary error transport (3011 occurrences)
    • tfdiags.DiagnosticsAsError — wraps a Diagnostics slice as a single error for stdlib interop
    • tfdiags.NonFatalError — marks errors that should not halt execution
    • states/statemgr.LockError — state locking failure with lock info
    • providercache.InstallerError — provider installation failure with per-provider details
    • experiments.UnavailableError, experiments.ConcludedError — typed experiment registry errors
    • backend/remote.NonRetryableError — sentinel for retry loop termination
  • Wrapping approach: fmt.Errorf with %s (930 uses) is far more common than %w (234 uses). The codebase predates errors.Is/errors.As’s Go 1.13 introduction; only 56 uses of those functions exist. Error identity is typically checked by type assertion rather than unwrapping.
  • The Diagnostics pattern: The key design decision is to return tfdiags.Diagnostics (a slice) rather than error (a single value). This enables a single graph walk to accumulate 50+ resource errors simultaneously (diags.Append(...) — 4136 occurrences) and present all of them to the user rather than failing on the first. Each diagnostic carries HCL source location (file, line, column).
  • Examples:
    • internal/terraform/context.go — Plan() returns (plan, tfdiags.Diagnostics); callers call diags.HasErrors() then handle individually
    • internal/schemarepo/loadschemas/plugins.go:60 — lower-level functions still return bare fmt.Errorf("unavailable provider %q", addr.String()) — consistent with the boundary pattern

Configuration pattern#

  • Approach: Struct-based configuration with manual constructor wiring. Functional options are rare (one type Option func(*parserConfig) in internal/configs/parser_file_matcher.go). The dominant pattern is a single large config struct populated at boot time.
  • Example: terraform.ContextOpts is the primary config struct for the core engine:
    type ContextOpts struct {
    Providers    map[addrs.Provider]providers.Factory
    Provisioners map[string]provisioners.Factory
    Hooks        []Hook
    Parallelism  int
    Meta         *ContextMeta
    // ...
}
    command.Meta is populated once in initCommands() and value-copied into every command struct.
  • Assessment: The struct-based approach works well for a CLI tool where configuration is read once at startup. The value-copy of Meta into 50+ command structs is an unusual pattern that avoids shared mutable state between commands — each command gets its own snapshot.

Dependency injection#

  • Approach: Manual wiring — no framework (no wire, dig, or fx)
  • Evidence:
    • main.go:realMain() — sequential manual construction: cliconfig → disco → credsSrc → providerSrc → backendInit.Init() → initCommands(ctx, wd, streams, config, services, providerSrc, ...)
    • initCommands() constructs a single command.Meta value, then wraps it in cli.CommandFactory closures; each factory value-copies the Meta at invocation time
    • terraform.NewContext(opts) receives all dependencies (provider factories, hooks, parallelism) through ContextOpts — no global singletons in the core engine
    • Provider factories are type Factory func() (Interface, error) — lazy constructors stored in a map, resolved at walk time
  • Assessment: The manual approach is appropriate for a project that explicitly does not want to be embedded as a library. The Factory pattern for providers is well-suited to the lazy, on-demand subprocess lifecycle.

Other notable patterns#

GraphTransformer Pipeline (Composable Graph Construction)#

The transform pipeline is the most architecturally distinctive Go pattern in Terraform.

type GraphTransformer interface {
    Transform(*Graph) error
}

BasicGraphBuilder holds a Steps []GraphTransformer slice. Each operation (plan, apply, import, etc.) defines its own ordered list of ~15 transformers that progressively build the DAG. A graphTransformerMulti wraps a slice of transformers into a single step. The pipeline can be traced with GRAPH_TRACE log level.

  • Example: internal/terraform/graph_builder_apply.go:105 — ApplyGraphBuilder.Steps() returns ~17 transformer steps including OrphanResourceInstanceTransformer, CreateBeforeDestroyTransformer, ReferenceTransformer, etc.
  • Assessment: Highly composable and testable — each transformer can be unit-tested in isolation by building a minimal graph and applying just that step. The separation of construction (transformers) from execution (walker) is clean.

NilHook Embedding (Partial Interface Implementation)#

The Hook interface has ~20 methods (pre/post callbacks for every resource lifecycle event). Rather than forcing every implementer to write no-op stubs:

// NilHook is a Hook implementation that does nothing.
// Embed this in your own Hook implementation to get no-op defaults.
type NilHook struct{}

Five implementations embed NilHook: StateHook, UiHook, CountHook, JsonHook, stackeval.TerraformHook. They override only the hooks they care about.

  • Assessment: Idiomatic Go pattern for partial interface implementation. The comment on the Hook interface explicitly documents this idiom. The stopHook does NOT embed NilHook — it explicitly implements every method with return h.hook() to ensure the stop signal is checked on every callback.

Compile-Time Interface Satisfaction Checks#

Extensively used throughout the codebase:

var _ backend.Backend = (*Remote)(nil)
var _ backendrun.OperationsBackend = (*Remote)(nil)
var _ terraform.Hook = (*StateHook)(nil)
  • Usage: 20+ occurrences in non-test files; found in backend/remote/backend.go, backend/local/backend.go, backend/local/hook_state.go, backend/remote-state/s3/backend.go, etc.
  • Assessment: Best-practice Go idiom. Particularly valuable here because many types implement multiple interfaces simultaneously (Remote implements Backend, OperationsBackend, and Local at once). The checks ensure refactoring doesn’t silently break interface contracts.

Factory Map Registry#

Two distinct registry patterns:

  1. Command registry (commands.go:122) — Commands = map[string]cli.CommandFactory{...} with 50+ entries; each factory is a closure capturing meta by value
  2. Provider/backend registries — map[addrs.Provider]providers.Factory and map[string]backendInit.InitFn
  • Assessment: Simple and effective for static registrations. The command registry is intentionally a package-level var to allow test replacement (main_test.go:27 — Commands = make(map[string]cli.CommandFactory)).

Table-Driven Tests#

  • Prevalence: Very heavy — 603 occurrences of testCases, tt.name, or tests := []struct{...}
  • Style: Anonymous struct slices are the dominant form; named sub-tests via t.Run(tc.name, ...)
  • Example: internal/terraform/context_plan_test.go — nearly every test function is a table-driven suite exercising plan behavior across many HCL input variations

Atomic Stop Flag (Non-Context Halt)#

internal/terraform/hook_stop.go uses atomic.Uint32 (zero = running, nonzero = halt):

type stopHook struct {
    stop uint32
}
func (h *stopHook) hook() (HookAction, error) {
    if atomic.LoadUint32(&h.stop) != 0 {
        return HookActionHalt, nil
    }
    return HookActionContinue, nil
}

This is distinct from context.Context cancellation — it signals the graph walker to stop dispatching new work via the Hook mechanism, not by canceling the current context. This allows the current in-flight provider RPC to complete before halting.

Generics Usage (Go 1.18+, Selective)#

Generics appear exclusively in internal/promising/:

  • Promise[T any], PromiseResolver[T any], PromiseGet[T any] — generic future type for the Stacks async evaluator
  • Once[T any] — generic idempotent lazy evaluation (a deadlock-detecting sync.Once)
  • PromiseResolverList[T any], ptrSet[T any] — supporting types

The rest of the codebase uses no generics; the main codebase predates Go 1.18 and has not been retrofitted.

Type Switches for DAG Vertex Capabilities#

Graph nodes are stored as dag.Vertex (interface{}). Capabilities are discovered by type assertion:

if en, ok := v.(GraphNodeExecutable); ok {
    en.Execute(ctx, walk)
}
if dn, ok := v.(GraphNodeDynamicExpandable); ok {
    dn.DynamicExpand(ctx)
}
  • Usage: 22,719 type assertion occurrences in the repo (including generated code). The pattern of testing for optional interfaces via type assertion (rather than requiring all vertices to implement all interfaces) is central to the graph node design.
  • Assessment: An intentional design choice enabling additive extensibility — new capabilities can be added to graph nodes without modifying the walker. The GraphNodeDynamicExpandable interface for count/for_each expansion is the most architecturally significant example.
Backward Terraform — Architecture Terraform — Interfaces Forward