Syncthing — Architecture#

Architectural style#

Supervisor-tree daemon with a layered library core

Syncthing is a long-running daemon (not a request-response service) organized as a hierarchy of supervised services inspired by Erlang/OTP. Every major subsystem — the sync engine, connection manager, discovery system, REST API, event bus, config watcher, and upgrade checker — implements suture.Service and is registered under a single root *suture.Supervisor. If any service panics or returns an error it does not understand, suture restarts it; fatal errors propagate upward and shut down the application with a structured exit code.

The application code is structured in three tiers:

  1. Library (lib/, internal/) — 40+ packages providing all domain logic
  2. Assembly (lib/syncthing) — wires library packages together into a running App
  3. Binary (cmd/syncthing) — thin CLI shell that parses flags and calls lib/syncthing.New()/Start()

This separation means lib/syncthing.App can be embedded in other programs (third-party GUI wrappers do exactly this) without touching cmd/.

Component diagram (textual)#

cmd/syncthing/main.go
  └─ lib/syncthing.App            ← root supervisor (suture)
       ├─ ur.FailureHandler        ← crash/panic reporter
       ├─ db.DBService             ← SQLite maintenance goroutine
       ├─ auditService             ← optional audit log
       ├─ model.Model              ← core sync engine (LARGEST component)
       │    ├─ folderRunners[]     ← one supervised runner per folder
       │    │   ├─ sendRecvFolder  ← pull blocks, write files, emit events
       │    │   ├─ sendOnlyFolder  ← scan & push only
       │    │   └─ receiveOnly     ← accept remote files only
       │    └─ progressEmitter     ← download progress events
       ├─ discover.Manager         ← discovery aggregator
       │    ├─ local: beacon       ← UDP broadcast/listen
       │    └─ global: HTTPS       ← discovery server query
       ├─ connections.Service      ← transport manager
       │    ├─ QUIC listener/dialer
       │    ├─ TCP listener/dialer
       │    ├─ relay dialer        ← for NAT-blocked devices
       │    └─ nat.Service         ← UPnP/NAT-PMP port mapping
       ├─ ur.Service               ← anonymous usage reporting
       ├─ model.FolderSummaryService  ← folder completion events
       └─ api.Service              ← REST API + embedded web UI
            └─ events subscriptions (defaultSub, diskSub)

internal/db (SQLite)  ←─── used by model.Model directly
lib/protocol           ←─── used by connections.Service + model.Model
lib/fs                 ←─── used by scanner + folder runners
lib/events.Logger      ←─── shared bus used by all components
lib/config.Wrapper     ←─── shared config used by all components

Core components#

App (lib/syncthing)#

  • Package: github.com/syncthing/syncthing/lib/syncthing
  • Responsibility: Application root. Creates the suture supervisor, instantiates all components, wires dependencies, provides Start()/Stop()/Wait() lifecycle.
  • Key types: App, Options, Internals
  • Dependencies: All major lib/ packages. This is the only package that imports everything else.

Model (lib/model)#

  • Package: github.com/syncthing/syncthing/lib/model
  • Responsibility: The heart of the application. Owns the global file index (reads/writes via internal/db), maintains per-folder runners (scanner + puller), manages active protocol connections (AddConnection), implements the protocol.Model callback interface (handles Index, IndexUpdate, Request, ClusterConfig, DownloadProgress messages from peers), and drives file transfers.
  • Key types: Model (interface, ~30 methods), model (concrete struct embeds *suture.Supervisor), folderRunners (serviceMap of sendRecvFolder/sendOnlyFolder/receiveOnlyFolder), ProgressEmitter
  • Dependencies: lib/config, lib/protocol, lib/fs, lib/scanner, lib/ignore, lib/events, lib/versioner, lib/stats, lib/semaphore, internal/db

Protocol (lib/protocol)#

  • Package: github.com/syncthing/syncthing/lib/protocol
  • Responsibility: Implements the Block Exchange Protocol (BEP). Each Connection wraps a TLS stream and provides typed message methods (Index, IndexUpdate, Request, ClusterConfig). Handles encryption for “untrusted” relay peers. Protobuf-encoded messages from proto/bep/.
  • Key types: Connection (interface), ConnectionInfo (interface), Model (callback interface that model.Model satisfies), RequestResponse, DeviceID, FileInfo, BlockInfo
  • Dependencies: proto/bep (generated protobuf), stdlib crypto/TLS

Connections Service (lib/connections)#

  • Package: github.com/syncthing/syncthing/lib/connections
  • Responsibility: Transport layer. Listens for and dials outgoing connections over QUIC, TCP, and relay transports. On successful TLS handshake + BEP Hello, calls model.AddConnection(conn, hello). Monitors connection health. Exposes discover.AddressLister so the discovery manager can advertise its listen addresses.
  • Key types: Service (interface), service (concrete, embeds *suture.Supervisor), dialerFactory, listenerFactory, genericDialer, genericListener
  • Dependencies: lib/model, lib/protocol, lib/discover, lib/nat, lib/events, lib/config

Discovery Manager (lib/discover)#

  • Package: github.com/syncthing/syncthing/lib/discover
  • Responsibility: Aggregates multiple discovery mechanisms. Local discovery uses UDP broadcasts via lib/beacon; global discovery queries the Syncthing discovery HTTPS server using device certificates for authentication. Returns []string of addresses for a given device ID on demand.
  • Key types: Manager (interface), Finder (interface), FinderService (interface, embeds Finder + suture.Service), AddressLister (interface)
  • Dependencies: lib/beacon, lib/config, lib/events, lib/protocol

Events Logger (lib/events)#

  • Package: github.com/syncthing/syncthing/lib/events
  • Responsibility: Typed, async in-process event bus. Components emit events (DeviceConnected, FolderScanCompleted, StateChanged, etc.) using bitfield-masked event types. The REST API subscribes and long-polls for events over HTTP. Subscriptions receive events over channels.
  • Key types: Logger (interface, embeds suture.Service), Subscription (interface), BufferedSubscription (interface), EventType (bitmask), Event
  • Dependencies: none except stdlib

Config Wrapper (lib/config)#

  • Package: github.com/syncthing/syncthing/lib/config
  • Responsibility: Loads, validates, migrates, and live-reloads XML configuration. Notifies all registered Committer subscribers (model, connections, API, etc.) when configuration changes via the Modify() transactional API. Implements suture.Service for its own queue-based serialization.
  • Key types: Wrapper (interface), Configuration (struct with JSON/XML tags), Committer (interface satisfied by all subscribers)
  • Dependencies: lib/protocol, lib/events

REST API (lib/api)#

  • Package: github.com/syncthing/syncthing/lib/api
  • Responsibility: HTTP server serving the AngularJS web UI (embedded as static assets) and a REST+JSON API. All GUI and CLI operations go through this layer. Bridges the model, discovery manager, connections service, and events into HTTP handlers.
  • Key types: Service (interface, implements suture.Service), service (concrete)
  • Dependencies: lib/model, lib/connections, lib/discover, lib/events, lib/config, lib/assets

Database (internal/db)#

  • Package: github.com/syncthing/syncthing/internal/db
  • Responsibility: SQLite-backed storage for file metadata, folder state, device statistics, and misc key-value data. Provides the DB interface and DBService for background maintenance. Supports both mattn/go-sqlite3 (cgo) and modernc.org/sqlite (pure Go) via build tags.
  • Key types: DB (interface), DBService (interface), Typed (typed key-value overlay), MiscDB
  • Dependencies: SQLite driver (build-tag selected)

Data flow#

File sync: local change → remote peer receives it#

1. Filesystem watcher (lib/watchaggregator)
      detects inotify/FSEvents change on a watched folder path
      → debounces + deduplicates events
      → notifies folder runner in model

2. Folder runner (model.sendRecvFolder / sendOnlyFolder)
      triggers lib/scanner on the changed path(s)
      scanner hashes file content into 128 KiB blocks (SHA-256)
      produces protocol.FileInfo{Name, Modified, Blocks[]}

3. model.Model
      updates local index in internal/db (SQLite)
      sends IndexUpdate message to all connected devices
      via protocol.Connection.IndexUpdate()

4. protocol.Connection (TLS/QUIC stream)
      marshals protobuf IndexUpdate message
      writes to peer over network

5. Remote model receives IndexUpdate callback
      updates its global index for the sending device
      computes "need" set: files in global index not yet local
      folder runner schedules blocks to pull

6. Remote folder runner
      sends Request messages to source device for each needed block
      (Request: folder, name, offset, size, hash)
      source model handles Request via model.Request() →
        reads block from local fs → returns bytes

7. Remote folder runner
      accumulates blocks into temporary file
      verifies each block hash
      atomic rename to final path (lib/osutil)
      updates local index in internal/db

8. Events emitted throughout:
      FolderScanProgress, RemoteIndexUpdated,
      ItemStarted, ItemFinished, FolderCompletion

Incoming connection flow#

connections.Service (listener goroutine)
  → TLS accept (QUIC or TCP)
  → verify peer certificate (extract DeviceID)
  → BEP Hello exchange
  → check DeviceID against config (known device?)
  → create protocol.Connection
  → model.AddConnection(conn, hello)
     → store conn in model.connections map
     → emit DeviceConnected event
     → send ClusterConfig to peer
     → send full Index for each shared folder

Initialization / Bootstrap#

The startup sequence in lib/syncthing.App.startup() (manual DI, no framework):

1. App.Start()
   → creates root suture.Supervisor ("main")
   → starts supervisor in background

2. App.startup()
   a. Add ur.FailureHandler   (crash reporter)
   b. Add db.DBService        (SQLite maintenance)
   c. Add auditService        (if audit writer configured)
   d. Create event subscriptions for API (defaultSub, diskSub)
   e. Maximize OS open-file limit
   f. Derive myID from TLS certificate bytes
   g. DB cleanup: remove entries for folders no longer in config
   h. Version detection: compare prevVersion in DB to build.Version
      → if upgrade detected, optionally drop delta indexes
   i. Run globalMigration(sdb, cfg) for schema migrations
   j. Create model.NewModel(cfg, myID, sdb, ...)
      → Add model to supervisor
   k. Configure TLS (TLS 1.3 only, cert, BEP ALPN, no session tickets)
   l. Create lateAddressLister (breaks chicken-and-egg: discover ↔ connections)
   m. Create discover.Manager(myID, cfg, cert, evLogger, addrLister)
      → Add to supervisor
   n. Create connections.Service(cfg, myID, model, tlsCfg, discoverer, ...)
      → Add to supervisor
   o. Wire addrLister.AddressLister = connectionsService
   p. Create ur.Service (usage reporting)
      → Add to supervisor
   q. setupGUI():
      → Create model.FolderSummaryService → Add
      → Create api.New(...) → Add
      → api.WaitForStart() blocks until HTTP server is listening
   r. Emit events.StartupComplete

3. All services now running concurrently under suture supervision

Dependency injection pattern: Fully manual. All constructors take explicit interface parameters. No wire, dig, or fx. The assembly code in startup() is the single place where all wiring happens, making the dependency graph easy to trace.

Bootstrap circularity resolution: A lateAddressLister wrapper is used to break the circular dependency between discovery (needs listen addresses) and connections (needs discovery to find peers). The wrapper is created empty, both services are constructed referencing it, then it is back-filled with the real AddressLister after both exist.

Configuration#

  • Format: XML files (config.xml) loaded by lib/config
  • Location: Platform-appropriate defaults via lib/locations (e.g., ~/.config/syncthing/, %APPDATA%\Syncthing\)
  • CLI flags: Parsed via github.com/alecthomas/kong (a flag-like DSL). Flags for --config, --data, --home override lib/locations defaults. Environment variables: STCONFDIR, STDATADIR, STHOMEDIR.
  • Live reload: config.Wrapper.Modify(modifyFn) serializes changes through a queue, then notifies all Committer subscribers. This means folder additions/removals, device changes, and option tweaks apply at runtime without restart in most cases.
  • Migration: Each major version may include migration functions run at startup via globalMigration(sdb, cfg). Config schema versioning is embedded in the XML.
  • No Viper: Configuration is fully custom XML-based, predating Viper’s adoption in the ecosystem.

Key design decisions#

1. Supervisor tree (suture) for fault isolation#

Every service implements suture.Service (Serve(ctx context.Context) error). The root supervisor will restart individual services that return transient errors, or shut down if a FatalErr propagates. This is unusual in Go but gives Syncthing Erlang-like resilience: a scanner crash doesn’t take down the API or active transfers.

2. BEP over QUIC/TLS as the transport#

The Block Exchange Protocol is a first-class architectural concern, defined in protobuf (proto/bep/), implemented in lib/protocol, and carried over QUIC (primary) or TCP fallback. QUIC gives multiplexed streams without head-of-line blocking, essential for concurrent block transfers. TLS 1.3 is mandated; mutual certificate authentication is used for device identity (no passwords, no PKI).

3. Interface-first component boundaries#

Every major component is accessed through an interface (model.Model, connections.Service, discover.Manager, events.Logger, config.Wrapper, protocol.Connection). Concrete structs are unexported. This enables clean substitution in tests and allows third-party consumers of lib/ packages to mock components they don’t own.

4. lib/syncthing assembly package as an embeddable unit#

The decision to put the wiring logic in lib/syncthing rather than cmd/syncthing/main.go means the entire running application can be embedded in any Go program. Third-party GUI wrappers (e.g., Syncthing-macOS, Syncthing-GTK) use this. The cmd/ layer is intentionally trivial.

5. Per-folder runners under the model supervisor#

Rather than having one global pull/push loop, each configured folder gets its own supervised service (a sendRecvFolder, sendOnlyFolder, or receiveOnlyFolder). This provides folder-level parallelism, fault isolation (a stuck folder doesn’t block others), and a natural place to attach folder-specific concurrency controls (IO limiter, semaphore). Adding or removing a folder at runtime is a matter of adding or stopping a single service in the folderRunners map.

Backward Syncthing — Dependencies Syncthing — Patterns Forward