Syncthing — Testing#

Test metrics#

  • Test files: 141
  • Source files (non-test): ~407 (548 total Go files − 141 test files)
  • Ratio (test files / source files): ~0.35 — roughly one test file per three source files
  • Test frameworks: stdlib testing only — no testify, gomock, ginkgo, or any third-party assertion library

Test organization#

  • Placement: Same package (package model, package fs, etc.) — all test files share the production package namespace, giving them access to unexported symbols. External _test packages are not used.
  • Helper packages:
    • lib/testutil — minimal shared helpers: BlockingRW (blocks on Read/Write until Close), NoopRW (silent discard), NoopCloser. Provides controlled blocking I/O for concurrency tests.
    • cmd/strelaysrv/testutil — relay-specific copy of the same blocking I/O helpers, kept local to avoid circular imports.
    • lib/model/testutils_test.go — large (~300 lines) in-package test setup file. Declares package-level testModel wrapper, newModel, setupModel, setupModelWithConnection, and a full init() that pre-initializes three protocol.DeviceIDs, default configs, and pre-wired *mocks.Connection stubs. Effectively a test fixture factory shared by the entire model package test suite.
    • lib/rc — integration test controller package. Provides rc.Process, which spawns a real syncthing binary via os/exec, connects to its REST API, and exposes a typed client (GetString, PostJSON, event polling). Used exclusively by the test/ integration suite. Not a mock — it talks to a live process.
    • Counterfeiter-generated mock packages: lib/model/mocks, lib/protocol/mocks, lib/events/mocks, lib/config/mocks, lib/connections/mocks, lib/discover/mocks — one package per interface boundary.
  • Fixtures:
    • lib/model/testdata/ — single tmpfile used by upgrade/scan tests.
    • lib/api/testdata/ and lib/config/testdata/ — static XML config files for config parsing and migration tests.
    • lib/fs/fakefs.go — 1026-line production-grade in-memory filesystem (see Fake Filesystem below). Not testdata files; it is a full Filesystem interface implementation registered under FilesystemTypeFake.

Test patterns#

Table-driven tests#

  • Prevalence: Heavy. 76 table-driven patterns found across *_test.go files.
  • Style: Named anonymous struct slices (tests := []struct{ name string; ... }{...}) iterated with t.Run(tt.name, ...). The name field is always present and used as the sub-test label.
  • Example: lib/model/blockpullreorderer_test.go:24Test_chunk uses a 7-case table covering normal, edge, and degenerate inputs for the block-chunking algorithm. Each case has an args sub-struct and a want field; the body is a single reflect.DeepEqual comparison.
  • Assessment: Clean, idiomatic. Inline literal values keep data visible in the test file. Where inputs are complex (e.g., protocol block lists), slice literals with named fields are used rather than helper constructors, keeping each case self-contained.

Mocking approach#

  • Strategy: counterfeiter-generated full spy/stub structs, not gomock expectations. Each generated struct records every call (with argument capture), and provides *Stub func(...) fields to override specific methods. Test code calls f.MethodNameCalls(func(...) ...) to install stubs.
  • Generation: //go:generate go tool counterfeiter -o mocks/model.go --fake-name Model . Model directives at the top of key interface files. The generated code is committed to the repo and regenerated via go generate.
  • Usage in tests: lib/model/fakeconns_test.go defines fakeConnection, which embeds *protocolmocks.Connection and layers domain-specific behaviour on top: it overrides Request to serve from an in-memory fileData map, stubs Close to close a channel and call model.Closed, and records DownloadProgress messages in a local slice. This embedding-plus-override pattern separates the counterfeiter boilerplate from test-specific logic.
  • Protocol mocks: lib/protocol/mocks/connection.go — generated from the 20-method protocol.Connection interface. Used as *protocolmocks.Connection in both testutils_test.go (as device1Conn, device2Conn package-level vars) and in fakeConnection. DeviceIDReturns, ConnectionIDReturns, and CloseCalls are called in test helpers to wire up default behaviour.
  • Example flow: setupModelWithConnectionnewModel(t, w, myID, nil) constructs a real model with a real SQLite DB in t.TempDir()addFakeConn(m, device1, "default") creates a fakeConnection backed by protocolmocks.Connectionm.AddConnection(fc, protocol.Hello{}) registers it with the live model. The test then exercises the model with realistic concurrency while the “peer” is a controlled in-process stub.

Fake Filesystem#

  • Not a mock, a production-quality substitute: lib/fs/fakefs.go (1026 lines) implements the full Filesystem interface backed by in-memory maps. File contents are generated deterministically from a name-seeded PRNG (not stored in RAM), so even large fake files occupy only metadata. Configuration is via URL query parameters (?files=1000&insens=true&latency=10ms&content=true) passed to fs.NewFilesystem(FilesystemTypeFake, url).
  • Impact: Virtually all lib/model unit tests run against FilesystemTypeFake. No temp directories, no OS filesystem calls, no cleanup. Tests are fast, hermetic, and portable. This is the most significant testing infrastructure investment in the project.
  • Parameters: files=N pre-populates N random files; insens=true makes the FS case-insensitive; latency=D introduces simulated read latency; content=true makes written bytes readable back (non-deterministic). The same fake FS is used for benchmarks: lib/fs/casefs_test.go benchmarks CaseFS traversal over a 10 000-file fake root.

Integration tests#

  • Present: Yes, substantial — 15+ test files in test/.
  • How: Each test in test/ uses lib/rc to spawn 2–4 real syncthing processes with pre-baked config directories (h1/, h2/, h3/), then drives them entirely through the REST API. Tests assert sync convergence by polling folder completion status, comparing file hashes, or checking event streams.
  • Separation: //go:build integration tag on all files in test/. Not run in normal go test ./...; must be explicitly enabled with -tags integration. The CI workflow does not appear to run integration tests on every PR (they require a built binary).
  • Scenarios covered: TestSyncCluster (3-node cluster with multiple shared folders), TestConflict (concurrent write conflict detection), TestIgnore (.stignore patterns), TestSymlinks, TestFiletypes, TestReconnect (connection drop and resume), TestManypeers, TestParallelScan, TestDelayScan, TestOverride (send-only folder override). Coverage of the full distributed sync protocol at the process level.
  • Support infrastructure: test/util.go provides removeAll, generateFiles, dirs, waitForScan, waitForCompletion helpers shared across integration tests. The lib/rc.Process type handles process lifecycle: it parses log output to detect startup completion, subscribes to event streams, and exposes Stop() which calls POST /rest/system/shutdown.

Test quality observations#

What’s done well#

  • Fake filesystem is exemplary. Rather than mocking os calls or using afero, syncthing built a full Filesystem interface implementation with configurable behaviour. Tests exercise real file operations (scan, pull, versioning) against an in-memory FS with no flakiness from disk speed or cleanup races.
  • t.Context() integration throughout. 265 uses of t.Context() / t.TempDir() / t.Cleanup() across test files. Tests honour test deadlines end-to-end — a timed-out test cancels the model’s context, avoiding hangs. This is especially important for tests that start supervised goroutine trees.
  • In-package test helpers avoid interface bloat. By placing helpers in testutils_test.go (same package, not exported), syncthing avoids adding test-only methods to production interfaces. The testModel wrapper adds convenience methods (testCurrentFolderFile, testCompletion) that call production methods and handle errors with t.Fatal — only possible from the same package.
  • counterfeiter over manual fakes. Generated spy stubs mean that adding a method to an interface immediately causes test compilation to fail (the generated mock is out of date), forcing go generate before the PR can be merged. This catches interface evolution.
  • Two-tier test strategy is coherent. Unit tests (fake FS, counterfeiter mocks, in-process) run in seconds and cover algorithmic behaviour. Integration tests (real binary, real sync) run the full protocol stack. The boundary is clean: unit tests never spawn child processes; integration tests never use fake objects.

What could improve#

  • No parallel unit tests. t.Parallel() is not used in any model unit test, even though most tests operate on independent testModel instances. Running 50+ sequential model tests is slower than necessary.
  • Integration test CI gap. Integration tests are not run on every PR (they require a pre-built binary and are guarded by a build tag). A regression in the sync protocol could pass unit tests but fail integration tests without being caught before merge.
  • init() in testutils_test.go is an anti-pattern. Package-level device IDs and config wrappers are initialized once in init(), shared by all tests. Tests that mutate defaultCfgWrapper require careful cloning via newDefaultCfgWrapper(t). New contributors may not realize the global state risk; the test file has no comment warning about it.
  • No testing/fstest use. Go 1.16+ testing/fstest.MapFS offers a stdlib alternative for simple FS needs. Syncthing pre-dates this and built fakefs.go instead — the custom implementation is more powerful (latency simulation, case-insensitivity, deterministic content), but testing/fstest could serve simpler packages that only need fs.FS semantics.

Patterns worth emulating#

  1. Full interface implementation as a test doublefakefs.go is the gold standard for testing code that depends on a rich interface. Configure via constructor/URL instead of per-method stubs. Avoids the combinatorial explosion of mock configuration.
  2. t.Context() as the default context in all tests — every goroutine tree started by a test will be cancelled when the test ends (or times out). No leaked goroutines, no test framework teardown needed.
  3. In-package helper file pattern — one large testutils_test.go per complex package that provides the full model/config/db setup. Keeps the actual test functions small and focused on the assertion, not the setup.
  4. counterfeiter + //go:generate at the interface definition site — self-documenting mock strategy: the interface file declares where the mock lives and how to regenerate it.