Usage: Core sync engine in fs/sync/sync.go. Two named goroutine pools — checkers and copiers — connected by buffered channel pipelines.
Example:fs/sync/sync.go:70 — toBeChecked *pipe and toBeUploaded *pipe fields on syncCopyMove. Pool sizes controlled by --checkers and --transfers CLI flags.
Assessment: Highly idiomatic. Separating IO-bound work (metadata checks) from bandwidth-bound work (uploads) at different concurrency levels is textbook Go pipeline design. The pipe type (fs/sync/pipe.go) is a custom unbounded queue that wraps a heap for optional --order-by sorting — elegant solution to the ordering problem without blocking.
Usage: 123 go func usages; 282 WaitGroup .Add/.Wait sites across the codebase.
Example:fs/sync/sync.go — the sync engine starts N checker goroutines reading from toBeChecked, N copier goroutines reading from toBeUploaded, each in a WaitGroup.
Assessment: Classic Go worker pool. --checkers and --transfers flags expose this to users, giving direct control over concurrency without any abstraction layer.
Usage: 5 files use golang.org/x/sync/errgroup, focused on fan-out within backends.
Example:backend/combine/combine.go:187 — g, gCtx := errgroup.WithContext(ctx) to fan out listing across multiple upstream remotes simultaneously. fs/operations/multithread.go:198 — parallel chunk upload.
Assessment: Correctly scoped: errgroup is used for bounded, same-scope fan-outs; longer-lived pipelines use explicit goroutines + channels. Clean separation of idiom by use case.
Example: March walker passes context through all list calls; cancelling the context at sync root propagates to all in-flight backend operations.
Assessment: Context threading is thorough and consistent. Every I/O call accepts a context. The additional pattern of fs.AddConfig(ctx, ci) / fs.GetConfig(ctx) extends context to carry config overrides — an elegant alternative to globals.
Usage:lib/atexit/atexit.go — a package-level registry of cleanup functions triggered by OS signals (SIGTERM, SIGINT).
Example:lib/atexit.Register(fn) returns a handle; lib/atexit.Run() is called in normal exit. OnError(&err, cancelFunc) convenience wraps a defer that also fires on error exit.
Assessment: Thoughtful design. sync.Once ensures handlers fire exactly once regardless of whether the signal path or normal exit path calls Run(). atomic.Int32 for state avoids mutex in hot signal path. The OnError helper is a useful idiom for deferred cancellation-on-error.
Usage: Nearly every backend instantiates a pacer.Pacer for API call rate limiting. Global bandwidth throttle via token bucket in fs/accounting/token_bucket.go.
Example:backend/drime/drime.go:377 — fs.NewPacer(ctx, pacer.NewDefault(pacer.MinSleep(...), pacer.MaxSleep(...), pacer.DecayConstant(...))). Each backend wraps every API call in f.pacer.Call(func() (bool, error) {...}).
Assessment: The shouldRetry(ctx, resp, err) closure pattern is consistent across all 70+ backends: return (bool, error) from pacer callbacks, delegate HTTP status code → retry decision to fserrors.ShouldRetryHTTP. The pacer handles exponential backoff with jitter, 429 responses, and transient errors transparently.
Usage: 92 select {} blocks throughout the codebase.
Example:lib/atexit/atexit.go — select on signal channel for shutdown. backend/ftp/ftp.go — select on result and error channels for async FTP responses.
Assessment: Conventional usage. No unusual patterns.
Style: Mixed — primarily fmt.Errorf with %w wrapping (1,601 uses) for context addition, plus a rich behavioral error type hierarchy in fs/fserrors for retry semantics.
Error types defined:
fserrors.retryError — string-typed error that satisfies Retrier interface; signals the operation should be retried
fserrors.wrappedRetryError / fserrors.wrappedFatalError / fserrors.wrappedNoRetryError — wrapper types that add retry/fatal/no-retry behavior to any existing error via Unwrap()
fserrors.FatalError(err), fserrors.NoRetryError(err), fserrors.RetryError(err) — constructors that decorate errors with retry policy
Per-backend API error types (e.g., backend/b2/api.Error, backend/webdav/api.Error) for HTTP API response parsing
Wrapping approach:fmt.Errorf("%w", err) is the dominant style. The fserrors package implements its own Unwrap() chain so errors.Is/errors.As work through decorated errors. errors.Is used in 132 places; errors.As in 27.
Key behavior:fserrors.ShouldRetry(err) walks the error chain via errors.As looking for Retrier, Fataler, and NoRetrier interfaces, and also checks context.Canceled and context.DeadlineExceeded. ShouldRetryHTTP checks HTTP status codes (429, 500, 502, 503, etc.) against a per-backend list.
Examples:
fs/fserrors/error.go:116 — FatalError(err) wraps any error with fatal semantics
backend/drime/drime.go:285 — shouldRetry is a local function per-backend that calls fserrors.ShouldRetry plus HTTP-specific check
fs/sync/pipe.go:35 — fserrors.FatalError(err) for unrecoverable --order-by parse errors
Approach: Custom config subsystem (no Viper/Cobra binding). Three layers:
INI-style ~/.config/rclone/rclone.conf parsed by fs/config/configfile
Environment variables (RCLONE_*) mapped via configflags
CLI flags bound to fs.ConfigInfo struct via pflag/cobra
Context-carried overrides:fs.AddConfig(ctx, &ConfigInfo{...}) returns a new context carrying a modified copy; fs.GetConfig(ctx) reads it back. This is the DI mechanism for per-operation config mutation without globals.
Backend config via configmap.Mapper: Backend NewFs() constructors receive a configmap.Mapper interface rather than a concrete struct. configstruct.Set(m, &opts) uses reflection to populate option structs from the mapper. Backend authors declare Options []fs.Option in their RegInfo; they never parse flags directly.
Functional options in libraries:lib/pacer and lib/http use functional options (type Option func(*pacerOptions); WithConfig, WithAuth, WithTemplate). This is scoped to library-level APIs, not the main config system.
Example:lib/pacer/pacer.go:50 — type Option func(*pacerOptions) with pacer.MinSleep(d), pacer.MaxSleep(d), pacer.DecayConstant(n) as Option constructors.
fs.GetConfig(ctx) — context as primary DI for config (avoids global mutation)
No DI framework (no Wire, Dig, or Fx)
init() plugin pattern: Each backend package’s init() calls fs.Register(&RegInfo{Name: "s3", NewFs: NewFs, Options: ...}). backend/all/all.go blank-imports all backends, triggering all init() functions. A custom build can swap backend/all/all.go to include only desired backends.
Assessment: The global registry is intentional and fits the CLI tool model — there is one process, one binary, no runtime plugin loading. context.Context as the DI mechanism for config is idiomatic and testable (tests call fs.AddConfig(ctx, ci) to override config without touching globals).
Features Struct with Function-Typed Fields (Optional Capabilities)#
The fs.Features struct (fs/features.go) holds ~50+ fields, some boolean flags and some function-typed optional operations (Purge func(...), Copy func(...), Move func(...), DirMove func(...), OpenChunkWriter func(...), Shutdown func(...), etc.).
Callers check if do := f.Features().Purge; do != nil { do(ctx, dir) } before invoking optional ops.
This replaces the classical Go pattern of many small single-method interfaces (Purger, Copier, etc.) and the required type assertions. Capability discovery is a single .Features() call.
Tradeoff: Features is a large, growing struct — every new capability adds a field.
Both backends and CLI commands register themselves via init() functions. The root program is 15 lines; the blank-import aggregators (backend/all/all.go, cmd/all/all.go) do all the wiring.
This is a deliberate architectural choice that keeps the core kernel free of any import of backends.
fs/sync/pipe.go implements an unbounded queue with an optional heap-based ordering for --order-by (size, name, modtime). It uses a chan struct{} as a semaphore/notification channel and a mutex-protected slice (or heap) as the backing store.
This hybrid avoids the classic tradeoff between channel backpressure and unbounded buffering — the queue is unbounded but ordered, and the notification channel ensures goroutines block when the queue is empty.
135 type switch uses, mostly for handling multiple concrete types that implement fs.DirEntry (which includes both fs.Object and fs.Directory), and for inspecting error types in fs/fserrors.
Minimal adoption: ptr[T any](t T) *T in several backends (azurefiles, s3, filescom) as a one-liner pointer helper; deref[T any] in s3; check[T comparable] for metadata testing; NewUsageValue[T] in fs/types.go.
No generic data structures in the core kernel — lib/cache uses any rather than generics, predating or avoiding generics for compatibility.
Extensively used: var _ Retrier = wrappedRetryError{error(nil)} patterns in fs/fserrors/error.go ensure wrapper types satisfy their intended interfaces at compile time.
52 sync.Once uses. Key examples: lib/atexit uses registerOnce to install the signal handler exactly once regardless of how many Register calls occur; exitOnce ensures cleanup runs exactly once whether triggered by signal or normal exit.