Grafana — Architecture#

Architectural style#

Plugin-based Layered Monolith undergoing migration to Kubernetes-inspired Resource API

Grafana’s backend is best described as a plugin-based layered monolith: a single process that hosts all domain services, wired together at startup via Google Wire DI, organized in well-defined layers, and extensible at runtime through a gRPC-based plugin system. It is not a microservice architecture — everything runs in one grafana binary.

However, the project is mid-migration toward a Kubernetes-inspired resource-oriented architecture. The existing “Legacy API” (/api/...) layer coexists with a new “Resource API” layer (/apis/...) that borrows Kubernetes API conventions (URL structure, versioning, schema, namespacing, watch semantics). The long-term goal is for Resource APIs to become the sole interface, deprecating the Legacy API entirely.

Evidence:

pkg/server/wire_gen.go (1939 lines, generated) wires hundreds of services into a single *Server
pkg/registry/registry.go defines BackgroundService — the universal lifecycle interface every service implements
contribute/architecture/k8s-inspired-backend-arch.md explicitly documents the dual-API architecture and migration plan
apps/ directory contains ~20 standalone, Kubernetes-style resource apps (each is its own Go module)

Component diagram (textual)#

┌─────────────────────────────────────────────────────────────────┐
│                    grafana binary (single process)               │
│                                                                  │
│  ┌─────────────────────────────────────────────────────────┐    │
│  │                    pkg/api (HTTPServer)                  │    │
│  │  ┌────────────────────┐  ┌────────────────────────────┐ │    │
│  │  │  Legacy API        │  │  Resource API              │ │    │
│  │  │  /api/...          │  │  /apis/<group>/<version>/  │ │    │
│  │  │  (pkg/api/api.go)  │  │  (pkg/apiserver/ embed)    │ │    │
│  │  └────────┬───────────┘  └────────────┬───────────────┘ │    │
│  └───────────┼─────────────────────────── ┼────────────────┘    │
│              │                             │                      │
│  ┌───────────▼─────────────────────────── ▼────────────────┐    │
│  │                  pkg/services/ (~60 packages)            │    │
│  │  alerting  dashboards  auth  users  LDAP  secrets  …    │    │
│  │           ↕ in-process bus (pkg/bus/)                    │    │
│  └─────────────────────────┬───────────────────────────────┘    │
│                             │                                     │
│  ┌──────────────────────────┼──────────────────────────────┐    │
│  │           pkg/infra/ (Infrastructure Layer)              │    │
│  │  ┌──────────┐ ┌───────┐ ┌──────────────┐ ┌──────────┐  │    │
│  │  │  infra/  │ │infra/ │ │ infra/       │ │ infra/   │  │    │
│  │  │  db/     │ │log/   │ │ remotecache/ │ │ tracing/ │  │    │
│  │  │ sqlstore │ │       │ │ localcache/  │ │(OTel)    │  │    │
│  │  └──────────┘ └───────┘ └──────────────┘ └──────────┘  │    │
│  └────────────────────────────────────┬─────────────────────┘   │
│                                        │                          │
│  ┌─────────────────────────────────────┼──────────────────────┐  │
│  │             Storage Layer           │                        │  │
│  │  ┌─────────────────────┐  ┌─────── ▼─────────────────────┐│  │
│  │  │  Legacy sqlstore    │  │  Unified Storage              ││  │
│  │  │  (per-resource      │  │  (pkg/storage/unified/)       ││  │
│  │  │   SQL tables)       │  │  implements k8s storage.      ││  │
│  │  └──────────┬──────────┘  │  Interface over SQL           ││  │
│  │             │             └────────────┬──────────────────┘│  │
│  └─────────────┼──────────────────────────┼───────────────────┘  │
│                └──────────────┬───────────┘                       │
│                               │                                   │
│                    ┌──────────▼──────────┐                       │
│                    │  SQL DB             │                       │
│                    │ (PostgreSQL/MySQL/  │                       │
│                    │  SQLite)            │                       │
│                    └─────────────────────┘                       │
│                                                                  │
│  ┌────────────────────────────────────────────────────────┐     │
│  │  pkg/plugins/ (Plugin System)                          │     │
│  │  discovery → loader → signature verify → gRPC backend  │     │
│  └────────────────────────────────────────────────────────┘     │
└─────────────────────────────────────────────────────────────────┘
          ↕ gRPC (protobuf)
┌─────────────────────────┐
│  External Plugin Process │
│  (data source / panel /  │
│   app plugin binary)     │
└─────────────────────────┘

Core components#

HTTPServer (API Gateway)#

Package: pkg/api/
Responsibility: HTTP routing, middleware chain, request handling for both Legacy API (/api/...) and proxying/integration with the embedded Kubernetes-style API server (/apis/...). Hosts the health, metrics, pprof, and swagger endpoints.
Key types: *api.HTTPServer, routing.RouteRegister
Dependencies: All service packages, middleware, setting, plugins

Service Layer#

Package: pkg/services/ (~60 sub-packages)
Responsibility: All domain business logic. Each sub-package owns one domain: ngalert (alerting), dashboards, auth, authn, users, orgs, accesscontrol, secrets, featuremgmt, apiserver, etc.
Key types: Each service defines an interface (e.g. dashboards.DashboardService) in the same package; implementations live alongside. Wired into the server via Wire DI.
Dependencies: pkg/infra/, other services via interfaces, pkg/bus/

Infrastructure Layer#

Package: pkg/infra/
Responsibility: Cross-cutting primitives with no domain knowledge: structured logging (log), metrics/Prometheus (metrics), database access (db, sqlstore), distributed cache (remotecache), local in-memory cache (localcache), HTTP client with observability (httpclient), distributed key-value store (kvstore), OpenTelemetry tracing (tracing), feature flags (featuremgmt), server lock for HA (serverlock)
Key types: db.DB, log.Logger, tracing.Tracer, remotecache.CacheStorage, httpclient.Provider
Dependencies: External: Prometheus, OpenTelemetry, Redis/Memcached (optional), SQL driver. No domain dependencies.

Plugin System#

Package: pkg/plugins/
Responsibility: Full plugin lifecycle: discovery (filesystem, CDN), loading, signature verification, Angular detection, gRPC backend process management (via grafana-plugin-sdk-go), plugin registry, and query routing.
Key types: plugins.Plugin, plugins.BackendPlugin, manager.PluginManager
Dependencies: pkg/infra/, github.com/grafana/grafana-plugin-sdk-go (SDK shared with external plugins)

Resource API Layer (new architecture)#

Packages: pkg/apiserver/, pkg/apis/, pkg/aggregator/, pkg/registry/apis/, pkg/registry/apps/
Responsibility: Embeds a Kubernetes-compatible API server into the Grafana process. Handles /apis/... requests using k8s.io/apiserver machinery. Resources can be defined via the Registry Approach (Go, pkg/registry/apis/) or the Apps Approach (CUE + App SDK, apps/).
Key types: APIGroupRunner, APIRegistrar, rest.Storage implementations
Dependencies: k8s.io/apiserver, k8s.io/apimachinery, pkg/storage/unified/

Unified Storage#

Package: pkg/storage/unified/
Responsibility: Single persistence abstraction for the Resource API. Implements k8s.io/apiserver/pkg/storage.Interface over Grafana’s SQL databases (no etcd required). Provides resource versioning via monotonic counters, optimistic concurrency control, Watch support via resource_history table, and optional gRPC backend for remote storage.
Key types: apistore.Storage, resource.ResourceClient
Dependencies: pkg/infra/db, SQL layer, optionally gRPC remote storage

Background Services / Module Manager#

Package: pkg/registry/, pkg/modules/
Responsibility: Lifecycle management for all background services. Every service that does async work implements registry.BackgroundService (Run(ctx) error). The ManagerAdapter (via dskit’s ModuleManager) starts them all after Init() and coordinates graceful shutdown.
Key types: registry.BackgroundService, registry.BackgroundServiceRegistry, adapter.ManagerAdapter
Dependencies: github.com/grafana/dskit (shared infrastructure from Grafana Labs)

Configuration#

Package: pkg/setting/
Responsibility: INI-based configuration loading (conf/defaults.ini + conf/custom.ini + environment variable overrides). Supports hot-reload for some settings via SIGHUP. Also provides the setting.Cfg struct that is passed throughout the DI graph.
Key types: setting.Cfg, setting.OSSImpl
Dependencies: gopkg.in/ini.v1

Bus (Event Bus)#

Package: pkg/bus/
Responsibility: In-process, synchronous publish/subscribe bus for domain events. Used primarily for cross-service communication that would otherwise create circular imports. Declining in use as services move to direct interface injection.
Key types: bus.Bus, bus.InProcBus
Dependencies: pkg/infra/tracing

Data flow#

Legacy API request (e.g., GET /api/dashboards/uid/:uid)#

HTTP client
  → pkg/middleware/ (auth, CSRF, session, logging, rate limit)
  → pkg/api/HTTPServer router (gorilla/mux or custom)
  → pkg/api/dashboard.go handler
  → pkg/services/dashboards.DashboardService.GetDashboard()
    → pkg/infra/db (sqlstore) SQL query
  ← result propagates back up
  → JSON response

Resource API request (e.g., GET /apis/dashboard.grafana.app/v1alpha1/dashboards/:name)#

HTTP client
  → pkg/middleware/ (auth, CSRF)
  → embedded kube-apiserver (pkg/apiserver/)
  → API Group handler (pkg/registry/apis/dashboard/ or apps/dashboard/)
  → pkg/storage/unified/ (Unified Storage)
    → k8s storage.Interface → ResourceClient → SQL (resource_history table)
  ← k8s-style response (JSON/protobuf with TypeMeta, ObjectMeta)

Plugin query (e.g., Prometheus data source query)#

Dashboard/Explore panel query
  → pkg/api/datasources.go (proxy handler)
  → pkg/plugins/manager QueryData()
    → gRPC call to plugin process (grafana-plugin-sdk-go protocol)
    → Plugin process queries Prometheus HTTP API
    → gRPC response with data frames
  → data frames serialized to frontend format
  ← JSON to browser

Initialization / Bootstrap#

Startup sequence (grafana server):

CLI parsing — urfave/cli/v2 parses flags in pkg/cmd/grafana-server/commands/cli.go
Config loading — setting.NewCfgFromArgs() reads INI files + env vars → *setting.Cfg
Feature flags init — featuremgmt.InitOpenFeatureWithCfg(cfg) bootstraps the OpenFeature SDK for dynamic flags
Wire DI — server.Initialize(ctx, cfg, opts, apiOpts) is called. This is the generated wire_gen.go:Initialize() — a 1939-line function that explicitly constructs every service in topological order, passing dependencies as constructor arguments. No reflection, no service locator.
Server.Init() — registers fixed RBAC roles and runs init-time provisioners (datasources, dashboards from YAML)
Tracing span — a root span is started for the server Run lifecycle
systemd READY — READY=1 notification sent to systemd (if running as a unit)
Background services start — ManagerAdapter.Run(ctx) starts all BackgroundService implementations concurrently. Each runs in its own goroutine, blocking on ctx.Done() for shutdown. Services include: HTTP server, alerting scheduler, provisioning poller, plugin store loader, stats collector, etc.
Signal handling — a goroutine listens for SIGTERM/SIGINT and calls Server.Shutdown(ctx, reason), which triggers ManagerAdapter.Shutdown() with a 30-second timeout.

Dependency Injection pattern: Manual Wire (github.com/google/wire). Wire reads the //go:build wireinject file (wire.go) and generates wire_gen.go. The injector signatures declare what the top-level *Server needs; Wire traces the full dependency graph at compile time. OSS vs Enterprise variants are split via build tags (oss/enterprise) in wireexts_oss.go / wireexts_enterprise.go.

Configuration#

Format: INI (.ini files) via gopkg.in/ini.v1
Files: conf/defaults.ini (shipped defaults) → conf/custom.ini (operator overrides, gitignored) → CLI flags / environment variables
Loading: setting.NewCfgFromArgs() → setting.Cfg struct. All services receive *setting.Cfg via Wire injection.
Hot-reload: SIGHUP triggers log.Reload() for logger reconfiguration. Some settings require restart.
Feature flags: Managed via pkg/services/featuremgmt/. Flags are defined in Go, generate constants via make gen-feature-toggles, and evaluated via OpenFeature SDK (supports dynamic flag providers in enterprise). Each flag can be enabled globally, by org, or by user.
Plugin config: pkg/plugins/config/ provides PluginManagementCfg and PluginInstanceCfg — separate from the main Cfg — wired via Wire.
No Viper: Grafana uses its own INI-based config system. Viper is not used.

Key design decisions#

1. Google Wire for compile-time DI at monolith scale#

Grafana’s dependency graph spans hundreds of services. Wire generates a single explicit function (Initialize in wire_gen.go) rather than using reflection-based containers. This catches circular dependencies and missing providers at build time. The 1939-line generated file is an artifact of scale, not a design flaw — it is never hand-edited. OSS/Enterprise variants are composed via build tags, not runtime checks.

2. `BackgroundService` as the universal service lifecycle interface#

Every long-running service implements a single Run(ctx context.Context) error method. The ManagerAdapter (wrapping grafana/dskit’s ModuleManager) starts all of them concurrently after Init. This uniform interface means new services integrate without changing the lifecycle orchestration. Graceful shutdown propagates through context cancellation.

3. Dual API architecture: Legacy `/api/...` + Kubernetes-style `/apis/...`#

Rather than rewriting the entire API surface at once, Grafana adopted a controlled migration path. The Resource API layer (/apis/...) uses embedded k8s.io/apiserver machinery, giving Grafana versioned, schema-validated, watch-capable APIs without adopting etcd or running a separate apiserver process. Feature flags gate the migration per resource type (Alpha → Beta → GA → deprecate Legacy → remove Legacy). This is documented in contribute/architecture/k8s-inspired-backend-arch.md.

4. gRPC-based plugin isolation#

All external plugins communicate with the Grafana backend over gRPC using a shared protocol defined in grafana-plugin-sdk-go. This provides:

Language-agnostic plugins (any language with gRPC)
Process isolation (plugin crashes don’t kill Grafana)
Version negotiation between host and plugin Some built-in data source plugins (e.g. CloudWatch, Azure Monitor) can also run as standalone gRPC processes via their standalone/ subdirectories, enabling independent deployment.

5. CUE as the schema source of truth for Grafana resources#

Dashboard, folder, alerting, and other resource types are defined first in CUE (kinds/, apps/*/kinds/*.cue). Code generation (make gen-cue) produces both Go structs and TypeScript types from the same schema. This eliminates the class of bugs where frontend and backend type definitions diverge. The Apps approach makes CUE-first the default for all new resources.

Grafana — Architecture#

Architectural style#

Component diagram (textual)#

Core components#

HTTPServer (API Gateway)#

Service Layer#

Infrastructure Layer#

Plugin System#

Resource API Layer (new architecture)#

Unified Storage#

Background Services / Module Manager#

Configuration#

Bus (Event Bus)#

Data flow#

Legacy API request (e.g., GET /api/dashboards/uid/:uid)#

Resource API request (e.g., GET /apis/dashboard.grafana.app/v1alpha1/dashboards/:name)#

Plugin query (e.g., Prometheus data source query)#

Initialization / Bootstrap#

Configuration#

Key design decisions#

1. Google Wire for compile-time DI at monolith scale#

2. BackgroundService as the universal service lifecycle interface#

3. Dual API architecture: Legacy /api/... + Kubernetes-style /apis/...#

4. gRPC-based plugin isolation#

5. CUE as the schema source of truth for Grafana resources#

2. `BackgroundService` as the universal service lifecycle interface#

3. Dual API architecture: Legacy `/api/...` + Kubernetes-style `/apis/...`#