Pop — Architecture#

Architectural style#

Layered Library with Dialect Plugin System

Pop is a library, not an application, so there is no long-running process or server. Its architecture is a shallow three-layer stack:

  1. User-facing API layer — Connection, Query, and the free functions on them (Find, Save, Where, etc.) form the public surface.
  2. Abstraction layer — two internal interfaces (dialect and store) decouple the ORM logic from both the underlying SQL driver and from database-specific SQL generation.
  3. Dialect/driver layer — five concrete dialect implementations (postgresql, mysql, mariadb, sqlite, cockroach) satisfy the dialect interface and translate generic operations into database-specific SQL.

The soda CLI is a thin wrapper that bootstraps the library and delegates entirely to its public API.

Component diagram (textual)#

┌─────────────────────────────────────────────────────────────┐
│                         soda CLI                             │
│  (soda/cmd — Cobra commands: create, drop, migrate, gen)    │
└──────────────────────────┬──────────────────────────────────┘
                           │ imports pop public API
┌──────────────────────────▼──────────────────────────────────┐
│                   Pop Library (root package)                  │
│                                                              │
│  ┌──────────────────┐   ┌────────────────────────────────┐  │
│  │   Config system  │   │      Connection                │  │
│  │  (config.go,     │──▶│  - Store store                 │  │
│  │   database.yml,  │   │  - Dialect dialect             │  │
│  │   envy env vars) │   │  - TX *Tx                      │  │
│  └──────────────────┘   └──────────┬───────────┬─────────┘  │
│                                    │           │             │
│                          ┌─────────▼──┐  ┌────▼──────────┐  │
│                          │   Query    │  │  dialect i/f  │  │
│                          │  (builder) │  │  crudable     │  │
│                          │  .Where()  │  │  fizzable     │  │
│                          │  .Order()  │  │  quotable     │  │
│                          │  .Limit()  │  └──────┬────────┘  │
│                          │  .ToSQL()  │         │           │
│                          └─────┬──────┘   ┌─────▼──────────┐│
│                                │          │  PostgreSQL    ││
│                          ┌─────▼──────┐   │  MySQL         ││
│                          │  store i/f │   │  MariaDB       ││
│                          │  (sqlx     │   │  SQLite        ││
│                          │  wrapper)  │   │  CockroachDB   ││
│                          └─────┬──────┘   └───────────────┘│
│                                │                            │
│  ┌────────────┐  ┌──────────┐  │ ┌──────────┐ ┌─────────┐  │
│  │  Model     │  │ Migrator │  │ │Assoc'ns  │ │Columns  │  │
│  │(reflection)│  │(file-    │  │ │(sub-pkg) │ │(sub-pkg)│  │
│  └────────────┘  │ based)   │  │ └──────────┘ └─────────┘  │
│                  └──────────┘  │                            │
└────────────────────────────────┼────────────────────────────┘
                                 │
                          ┌──────▼──────┐
                          │  sqlx/sql   │
                          │  (stdlib +  │
                          │  jmoiron)   │
                          └─────────────┘

Core components#

Connection#

  • Package: github.com/gobuffalo/pop/v6 (connection.go)
  • Responsibility: Central user-facing handle for all database interaction. Holds references to the active store (sqlx connection or transaction) and the dialect (database-specific behavior). All CRUD operations and query building originate from or flow through Connection.
  • Key types: Connection struct (Store store, Dialect dialect, TX *Tx)
  • Dependencies: store interface, dialect interface, internal/randx, internal/defaults, logging

dialect (interface + implementations)#

  • Package: github.com/gobuffalo/pop/v6 (dialect.go, dialect_*.go)
  • Responsibility: Abstracts all database-specific behavior. Composed of three sub-interfaces: crudable (SELECT/INSERT/UPDATE/DELETE), fizzable (fizz DSL translation), quotable (identifier quoting). Implementations also handle CreateDB, DropDB, DumpSchema, LoadSchema, and TruncateAll.
  • Key types: dialect (private interface), crudable, fizzable, quotable
  • Dependencies: gobuffalo/fizz (translator), columns sub-package, database/sql

store (interface + implementations)#

  • Package: github.com/gobuffalo/pop/v6 (store.go, db.go)
  • Responsibility: Thin abstraction over sqlx’s *sqlx.DB and *sqlx.Tx. Allows context propagation (via contextStore wrapper) without changing the core Connection API. Enables the instrumented driver overlay for tracing (Open Tracing, AWS X-Ray, etc.).
  • Key types: store (private interface), dB (wraps sqlx.DB), contextStore (wraps store + ctx), Tx
  • Dependencies: jmoiron/sqlx, database/sql, context

Query#

  • Package: github.com/gobuffalo/pop/v6 (query.go, clause.go, plus finders.go, executors.go, group.go, having.go, join.go, order.go, paginator.go)
  • Responsibility: Fluent query builder. Accumulates clauses (WHERE, ORDER, LIMIT, JOIN, GROUP BY, HAVING) and generates SQL via ToSQL(model). Delegates execution to the Connection.Store and dialect-specific CRUD methods.
  • Key types: Query struct, clause, clauses, operation (SELECT / DELETE)
  • Dependencies: Connection, Model, associations, logging

Model#

  • Package: github.com/gobuffalo/pop/v6 (model.go)
  • Responsibility: Reflection wrapper around user-provided structs. Derives table names (using gobuffalo/flect for pluralization), reads/writes ID, manages created_at/updated_at timestamps, and checks for optional interfaces (TableNameAble, TableNameAbleWithContext).
  • Key types: Model struct, Value (interface{}), TableNameAble, TableNameAbleWithContext
  • Dependencies: gobuffalo/flect, columns, gofrs/uuid

Migrator / FileMigrator#

  • Package: github.com/gobuffalo/pop/v6 (migrator.go, file_migrator.go)
  • Responsibility: Tracks and runs database migrations. FileMigrator reads .fizz or .sql migration files from disk, translates Fizz DSL through dialect.FizzTranslator(), records applied migrations in a schema_migration table, and supports up/down/status/reset operations.
  • Key types: Migrator, Migration, FileMigrator
  • Dependencies: dialect (for fizz translation and schema ops), Connection, gobuffalo/fizz

Config system#

  • Package: github.com/gobuffalo/pop/v6 (config.go, connection_details.go)
  • Responsibility: Loads and parses database.yml (with Go template expansion for env vars via gobuffalo/envy). Creates ConnectionDetails structs and registers them in the global Connections map. Supports multi-environment configs (development/test/production).
  • Key types: ConnectionDetails, ErrConfigFileNotFound
  • Dependencies: gobuffalo/envy, gopkg.in/yaml.v2, text/template

associations (sub-package)#

  • Package: github.com/gobuffalo/pop/v6/associations
  • Responsibility: Implements eager loading for BelongsTo, HasMany, HasOne, and ManyToMany associations via reflection on struct tags (has_many, belongs_to, has_one, many_to_many). Called by the Query’s eager-loading path.
  • Key types: Association (interface), AssociationSortable (interface), concrete association types
  • Dependencies: Root pop package (via circular reference at runtime via reflection, not import)

Data flow#

Typical read — c.Find(&user, id):

1. c.Find(&user, id)
   → Q(c).Find(&user, id)                         [finders.go]
   → q.Where("users.id = ?", id).First(&user)
   → q.ToSQL(model)                               [query.go]
       → sqlBuilder.compile() → "SELECT … FROM users WHERE users.id = ? LIMIT 1"
   → dialect.SelectOne(c, model, query)           [dialect_postgresql.go etc.]
   → c.Store.Get(&user, sql, args...)             [store: sqlx.DB.Get]
   → database/sql → network → PostgreSQL

Typical write — c.Save(&user):

1. c.Save(&user)                                  [executors.go]
   → c.ValidateAndSave or direct Save
   → model.beforeCreate(c) callback (if defined)
   → dialect.Create(c, model, columns)            [dialect_postgresql.go]
       → sqlBuilder for INSERT
   → c.Store.NamedExec(sql, &user)               [sqlx named params]
   → database/sql
   → model.afterCreate(c) callback (if defined)

Migration run — soda db migrate:

1. cobra PersistentPreRun: pop.LoadConfigFile()
2. migrate cmd: conn := pop.Connections["development"]
3. conn.Open()                                    [lazy connection]
4. migrator.Up()
   → scan migration files from directory
   → dialect.FizzTranslator().Translate(fizz)    [fizz DSL → SQL]
   → conn.RawQuery(sql).Exec()
   → insert into schema_migration

Initialization / Bootstrap#

The library uses lazy, config-file-driven initialization with no dependency injection framework:

  1. Package-level init() in config.go reads APP_PATH / POP_PATH env vars to prime lookup paths.
  2. LoadConfigFile() is called explicitly (by user code or soda’s PersistentPreRun). It: finds database.yml, template-expands env vars, YAML-unmarshals into map[string]*ConnectionDetails, calls NewConnection(deets) for each entry, and populates the global Connections map.
  3. NewConnection(deets) calls deets.Finalize() (resolves URL/dialect defaults), then looks up a dialect factory in the private newConnection map (registry pattern — each dialect_*.go file calls init() to register itself).
  4. Connect(env) or conn.Open()** opens the actual sqlx connection lazily (first time the connection is used).
  5. No DI framework: Dependencies are wired by direct struct assignment (c.Store = &dB{db}, c.Dialect = pg) inside constructors.

Configuration#

  • Primary mechanism: YAML file (database.yml) discovered by walking a list of lookupPaths (., ./config, ../, etc.)
  • Env var interpolation: The YAML file is first rendered as a Go text/template using gobuffalo/envy. Template functions envOr and env allow patterns like {{ envOr "DB_PASSWORD" "secret" }}.
  • CLI override: soda --config flag sets a custom config path; --env selects the environment key (defaults to $GO_ENV, then "development").
  • Connection pool config: Exposed directly in ConnectionDetails (Pool, IdlePool, ConnMaxLifetime, ConnMaxIdleTime).
  • Instrumented driver: UseInstrumentedDriver: true in ConnectionDetails wraps the driver with luna-duclos/instrumentedsql for distributed tracing.

Key design decisions#

  1. dialect interface as the portability seam. By composing crudable, fizzable, and quotable sub-interfaces, Pop achieves multi-database support without if-chains in query execution code. Each dialect file is fully self-contained; adding a new database requires only a new file implementing dialect and a registration call in init().

  2. store interface for context propagation without API breakage. Rather than adding a ctx context.Context parameter to every public method (which would be a breaking change), Pop wraps the sqlx connection in a contextStore that overrides every method to inject the context. Connection.WithContext(ctx) returns a copy with the wrapped store, enabling context propagation transparently.

  3. Registry pattern for dialect factories. The newConnection map (map[string]func(*ConnectionDetails) (dialect, error)) lets each dialect file register itself via init(). This avoids a central switch statement and allows build-tag-gated dialects (SQLite uses //go:build sqlite) without modifying core connection logic.

  4. Reflection-based model mapping (no code generation). Unlike GORM or sqlc, Pop derives table names, column names, and field mappings at runtime using reflection over struct tags (db:"column_name"). This eliminates a build step at the cost of runtime overhead and deferred error detection.

  5. Flat root package as the primary API. All public types (Connection, Query, Model, Migrator, etc.) live in the root package. This is a deliberate library-design choice: one import path gets everything. The trade-off — a large, dense package with 72 .go files and broad internal coupling — is accepted in exchange for a simpler consumer experience.

Backward Pop — Dependencies Pop — Patterns Forward