Vault — Architecture#

Architectural style#

Microkernel + Layered + Plugin-based

Vault is best described as a microkernel system with a layered architecture. The Core struct (vault/vault/core.go) acts as the central kernel — it owns every subsystem (router, barrier, token store, policy store, expiration manager, audit broker, identity store) and wires them together. Plugins (auth methods and secret engines) are mounted onto the kernel via a radix-tree router; they communicate with the kernel exclusively through the sdk/logical.Backend interface and never import kernel internals. The layers from bottom to top are:

  1. Physical storage (untrusted, durable) — pluggable backends in physical/
  2. Security barrier — AES-GCM encryption/decryption wrapping physical storage
  3. Core kernel — vault/vault/ manages routing, tokens, policies, leases, identity
  4. Backend plugins — builtin/credential/ (auth) and builtin/logical/ (secrets)
  5. HTTP layer — http/ translates REST requests to logical.Request objects
  6. CLI/Agent/Proxy — command/ interacts via the api/ client module

Evidence: core.go’s Core struct embeds all subsystems directly; sdk/logical.go’s Backend interface is the only contract between plugins and Core; the physical storage layer is referenced only through sdk/physical.Backend.

Component diagram (textual)#

┌─────────────────────────────────────────────────────────────────────┐
│  vault CLI / vault agent / vault proxy  (command/, api/)            │
└────────────────────────┬────────────────────────────────────────────┘
                         │ HTTPS (X-Vault-Token header)
┌────────────────────────▼────────────────────────────────────────────┐
│  HTTP Layer  (http/handler.go)                                      │
│  stdlib mux  ←  middleware chain (rate-limit, CORS, JSON-limits,   │
│                  request-priority, audit, forwarding)               │
└────────────────────────┬────────────────────────────────────────────┘
                         │ vault.Core.HandleRequest()
┌────────────────────────▼────────────────────────────────────────────┐
│  Core  (vault/vault/core.go, ~780 fields)                           │
│  ┌─────────────┐  ┌──────────────┐  ┌──────────────────────────┐   │
│  │  Router     │  │ TokenStore   │  │ ExpirationManager        │   │
│  │  (radix     │  │ (token       │  │ (lease TTLs, renewal,    │   │
│  │   tree)     │  │  lifecycle)  │  │  revocation)             │   │
│  └──────┬──────┘  └──────────────┘  └──────────────────────────┘   │
│  ┌──────▼──────┐  ┌──────────────┐  ┌──────────────────────────┐   │
│  │  PolicyStore│  │ IdentityStore│  │ AuditBroker              │   │
│  │  (ACL/HCL)  │  │ (entities,  │  │ (fan-out to audit        │   │
│  │             │  │  groups)     │  │  backends)               │   │
│  └─────────────┘  └──────────────┘  └──────────────────────────┘   │
└────────────────────────┬────────────────────────────────────────────┘
                         │ logical.Backend.HandleRequest()
┌────────────────────────▼────────────────────────────────────────────┐
│  Backend Plugins  (builtin/credential/, builtin/logical/)           │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────────────┐  │
│  │ Auth methods │  │Secret engines│  │  System backend          │  │
│  │ (aws, k8s,   │  │ (kv, pki,   │  │  (sys/ management API)   │  │
│  │  jwt, ldap…) │  │  db, ssh…)  │  │                          │  │
│  └──────────────┘  └──────────────┘  └──────────────────────────┘  │
└────────────────────────┬────────────────────────────────────────────┘
                         │ logical.Storage (BarrierView)
┌────────────────────────▼────────────────────────────────────────────┐
│  SecurityBarrier  (vault/vault/barrier_aes_gcm.go)                  │
│  AES-256-GCM encryption/decryption + keyring management            │
└────────────────────────┬────────────────────────────────────────────┘
                         │ physical.Backend
┌────────────────────────▼────────────────────────────────────────────┐
│  Physical Storage Backends  (physical/)                             │
│  Raft (default) │ Consul │ DynamoDB │ S3 │ Postgres │ etcd │ …     │
└─────────────────────────────────────────────────────────────────────┘

Core components#

Core (central kernel)#

  • Package: github.com/hashicorp/vault/vault
  • Responsibility: Orchestrates all Vault subsystems. Owns the mutable state of the running server — mounted backends, token store, policy store, leases, audit broker, identity store, cluster state. All HTTP requests ultimately reach Core.HandleRequest.
  • Key types: Core struct (~780 fields), CoreConfig (initialization config), MountTable (registered backends), Router (radix-tree path router)
  • Dependencies: All other packages depend on Core or are owned by Core; Core depends on sdk/logical, sdk/physical, audit, physical/raft, vault/seal, vault/quotas, vault/cluster, vault/eventbus

SecurityBarrier#

  • Package: github.com/hashicorp/vault/vault (barrier_aes_gcm.go, barrier.go)
  • Responsibility: Wraps the physical backend with transparent AES-256-GCM encryption. Every value written to storage is encrypted with a key from the in-memory keyring; the keyring itself is encrypted with the root key; the root key is protected by the seal mechanism (Shamir shares or auto-seal). All reads/writes from plugins go through a BarrierView that namespaces storage paths per backend mount.
  • Key types: SecurityBarrier interface (barrier.go), AESGCMBarrier struct (barrier_aes_gcm.go), BarrierView (barrier_view.go), Keyring (keyring.go)
  • Dependencies: sdk/physical.Backend (raw storage below), shamir/ (for Shamir unseal), vault/seal (for auto-seal)

Router#

  • Package: github.com/hashicorp/vault/vault (router.go)
  • Responsibility: Path-based dispatch to logical backends using a radix tree. When a request arrives for /v1/secret/foo, the router finds the mount entry for secret/, retrieves the associated logical.Backend, and calls HandleRequest. Manages tainted mounts, root paths, login paths, and binary paths.
  • Key types: Router struct (radix.Tree-backed), routeEntry (backend + mount metadata per route)
  • Dependencies: sdk/logical.Backend, github.com/armon/go-radix

HTTP Handler#

  • Package: github.com/hashicorp/vault/http
  • Responsibility: Translates HTTP requests (verb, path, headers, body) into logical.Request objects and sends them to Core. Implements a layered middleware chain: rate-limit quotas → JSON limits → token header size → request priority → CORS → help → core handler. Uses stdlib net/http mux for route registration.
  • Key types: HandlerProperties, HandlerAnchor, HandlerFunc; middleware wrappers are plain http.Handler decorators
  • Dependencies: vault.Core, sdk/logical, limits/, http/priority/

sdk/logical (Backend contract)#

  • Package: github.com/hashicorp/vault/sdk/logical
  • Responsibility: Defines the fundamental interfaces (Backend, Storage, Factory) that all plugins must implement. This is the boundary between Core and all pluggable backends — Core never imports plugin code directly, only logical.Factory functions.
  • Key types: Backend interface (7 methods), Storage interface (List/Get/Put/Delete), Request, Response, Secret, Auth, Factory func(context.Context, *BackendConfig) (Backend, error)
  • Dependencies: None outside stdlib + go-hclog (intentionally minimal)

Seal / Unseal subsystem#

  • Package: github.com/hashicorp/vault/vault/seal, github.com/hashicorp/vault/shamir
  • Responsibility: Controls access to the root key. On startup Vault is sealed — no data is accessible. Unsealing requires either providing Shamir key shares (combined via shamir/) or auto-unseal via an external KMS (AWS KMS, GCP CKMS, Azure Key Vault, etc.) wrapped by go-kms-wrapping. After unsealing, the root key decrypts the keyring, making the barrier operational.
  • Key types: Seal interface (vault/vault), autoSeal, SealConfig, shamir.Split/shamir.Combine
  • Dependencies: shamir/, go-kms-wrapping, sdk/physical

Expiration Manager#

  • Package: github.com/hashicorp/vault/vault (expiration.go)
  • Responsibility: Manages lease lifecycle — creation, renewal, and revocation. All dynamic secrets and auth tokens get a lease; the expiration manager uses a persistent lease index in barrier storage and a background worker pool (using helper/fairshare) to revoke expired leases.
  • Key types: ExpirationManager, leaseEntry, fairshare.WorkQueue
  • Dependencies: helper/fairshare, sdk/logical.Backend (for revocation calls back into plugins)

Plugin Catalog#

  • Package: github.com/hashicorp/vault/vault/plugincatalog
  • Responsibility: Registry of available plugins — both built-in (in-process) and external (out-of-process via hashicorp/go-plugin RPC). Maps plugin names/versions to factories. Built-in plugins from helper/builtinplugins/ are registered at startup.
  • Key types: PluginCatalog, BuiltinRegistry
  • Dependencies: sdk/plugin (go-plugin wrapper), helper/builtinplugins

Audit Broker#

  • Package: github.com/hashicorp/vault/audit
  • Responsibility: Fan-out of audit events (requests and responses) to all registered audit backends (file, socket, syslog). Uses a broker pattern: audit.Broker has multiple audit.Backend instances; every operation is logged to all enabled backends before the response is returned. If all audit backends fail, the request is rejected.
  • Key types: Broker, Backend interface, Entry (audit record), HeadersConfig
  • Dependencies: stdlib log, no external deps besides go-hclog

Data flow#

Typical authenticated secret read (GET /v1/secret/data/mykey):

1. TLS listener accepts TCP connection (command/server → http.Server)
2. http/handler.go middleware chain:
   a. priority.WrapRequestPriorityHandler — assign request priority
   b. wrapTokenHeaderSizeHandler — enforce token header size limit
   c. wrapMaxRequestSizeHandler — enforce body size limit
   d. rateLimitQuotaWrapping — check rate-limit quotas
   e. wrapJSONLimitsHandler — validate JSON depth/length
   f. wrapCORSHandler — CORS preflight
   g. wrapHelpHandler — help text fallback
3. http.ServeMux routes "/v1/" → handleRequestForwarding → handleLogical
4. handleLogical:
   a. buildLogicalRequest — parse HTTP into logical.Request (path, operation, data, token)
   b. core.HandleRequest(ctx, req)
5. Core.HandleRequest:
   a. checkToken — validate token via TokenStore, load policies
   b. AuditBroker.AuditRequest — write pre-request audit record
   c. checkPolicy — ACL check via PolicyStore
   d. router.Route(req) — find mount entry, dispatch to Backend
6. Backend.HandleRequest (e.g., kv secrets engine):
   a. Read from logical.Storage (BarrierView)
   b. BarrierView.Get → AESGCMBarrier.Get → physical.Backend.Get → Raft
   c. Decrypt value with active keyring key
   d. Return logical.Response{Data: ...}
7. Core.HandleRequest:
   a. AuditBroker.AuditResponse — write post-response audit record
   b. Return response to HTTP layer
8. http layer serializes logical.Response → JSON HTTP 200

Unseal flow (Shamir):

POST /v1/sys/unseal (with key share) ×N  →  Core.Unseal()
  → recordUnsealPart (accumulate shares)
  → shamir.Combine(shares) → combined root key
  → AESGCMBarrier.Unseal(rootKey) → decrypt keyring
  → postUnseal(): load mounts → setup credentials → setup policies
    → start expiration manager → load audits → load identity artifacts
  → Vault is now operational

Initialization / Bootstrap#

Startup sequence:

main.go → command.Run(args)
  → hashicorp/cli dispatches to *ServerCommand.Run()
  → Parse HCL config (command/server/config.go)
  → Initialize physical storage backend (e.g., raft.NewRaftBackend)
  → Create service registration (consul or kubernetes)
  → vault.NewCore(CoreConfig{
        Physical:           <storage backend>,
        HAPhysical:         <ha-capable backend if HA>,
        LogicalBackends:    <map[string]logical.Factory>,
        CredentialBackends: <map[string]logical.Factory>,
        AuditBackends:      <map[string]audit.Factory>,
        ...
    })
      → CreateCore (allocate Core struct)
      → coreInit (logger, metrics, cluster info)
      → mlock.LockMemory() (prevent swap)
      → NewAESGCMBarrier(physical) (create barrier — sealed)
      → configureLogicalBackends, configureCredentialsBackends
      → configureAuditBackends
      → NewQuotaManager
      → NewEventBus
  → http.NewVaultHandlerMux or http.Handler(core)
  → net.Listener on configured addresses
  → http.Server.Serve()
  → Wait for unseal (operator must provide key shares or autounseal fires)

Dependency injection pattern: Manual wiring. NewCore accepts a CoreConfig struct containing factory maps (not instances) for backends. Backends are instantiated lazily when first mounted. No wire, dig, or fx. The CoreConfig struct is the single injection point.

post-unseal setup runs a sequential slice of setup functions (buildUnsealSetupFunctionSlice): plugin catalog → mounts → credentials → quotas → expiration → audits → identity → MFA configs. Failure at any step re-seals Vault.

Configuration#

  • Format: HCL (HashiCorp Configuration Language), parsed by command/server/config.go using github.com/hashicorp/hcl
  • Key stanzas: storage {} (physical backend), listener "tcp" {} (network), seal {} (unseal mechanism), telemetry {}, api_addr, cluster_addr
  • Environment variables: VAULT_* env vars override or supplement config (e.g., VAULT_LOG_LEVEL, VAULT_API_ADDR, VAULT_CACERT)
  • CLI flags: vault server -config=<path> accepts multiple -config flags (merged)
  • Runtime config: Some settings live in barrier storage (policy definitions, mount table, audit config, token TTLs) and are loaded post-unseal; they persist across restarts.
  • No Viper: Vault uses its own HCL-based config library, not Viper. The internalshared/configutil package handles listener and telemetry config shared between server, agent, and proxy.

Key design decisions#

1. SecurityBarrier as the encryption boundary#

All data written to physical storage is encrypted. Plugins operate on BarrierView objects (namespaced, encrypted views), never raw storage. This means if the physical backend (e.g., a Consul KV store or S3 bucket) is compromised, the attacker gets only ciphertext. The barrier is unsealed at runtime by providing the root key, which is never persisted in plaintext. This is the foundational security guarantee of the entire system.

2. Seal/unseal lifecycle gates all operations#

Every request check (Core.HandleRequest) validates c.sealed. This means losing the root key (or all KMS access in auto-seal mode) makes Vault completely unavailable but guarantees confidentiality. The Shamir scheme distributes the key across N operators with a threshold of K — no single operator can unseal alone. This is implemented in-house (shamir/) rather than via a library, reflecting the criticality of the primitive.

3. sdk/logical.Backend as the universal plugin contract#

Auth methods, secret engines, the system backend, cubbyhole, identity — all are logical.Backend implementations. This single interface (7 methods) decouples Core from all plugin logic. Plugins receive only a BarrierView (namespaced storage) and a SystemView (read-only system info); they cannot read other plugins’ data or Core internals. External plugins communicate via gRPC (go-plugin), in-process plugins use direct function calls — the Core cannot tell the difference.

4. Radix-tree router enables path-based multi-tenancy#

The Router uses a radix tree of mount points (e.g., secret/, auth/ldap/, pki/) to dispatch requests. Mounts are dynamic: operators add/remove them at runtime via the sys/mounts API. The router also handles namespaces (enterprise) by prepending namespace paths. This design allows hundreds of independent secret engines to coexist in a single Vault cluster with O(log n) dispatch.

5. Three-module structure enables third-party plugin development#

sdk/ (github.com/hashicorp/vault/sdk) and api/ (github.com/hashicorp/vault/api) are published as independent Go modules. Plugin authors import sdk/logical to implement a Backend — they never touch vault/vault/. This is enforced by module boundaries: sdk has no replace directives pointing at the main module. The api/ module provides the Go client for operators writing automation or CLI tools without pulling in the server’s 300+ dependencies. The OpenBao fork exploited these clean boundaries successfully.

Backward Vault — Dependencies Vault — Patterns Forward