Pop — Testing#

Test metrics#

  • Test files: 60
  • Source files (non-test): ~107 (167 total .go files − 60 test files)
  • Ratio (test files / source files): ~0.56 — roughly one test file per source file, well above average for an ORM library
  • Test frameworks: testify/suite (for dialect-dispatched integration suites), testify/require (universal assertion library across all test files)

Test organization#

  • Placement: Split — pure unit tests use the _test package suffix (e.g., columns/columns_test.go is package columns_test) while integration tests use the same package (e.g., finders_test.go is package pop). The same-package placement gives integration tests access to unexported helpers like transaction() and ts().
  • Helper packages: No dedicated testutil/ or mock/ directory. Instead, helpers live inline in the root test package:
    • pop_test.go is the central test fixture file — it defines a large set of test model structs (User, Book, Song, CallbacksUser, etc.) with realistic db: tags and association annotations, making them double as both fixtures and documentation of the ORM’s struct contract.
    • transaction() helper wraps every integration test in a PDB.Rollback(func(tx *Connection)) call, ensuring each test starts from a clean database state without requiring teardown.
    • ts() helper translates a SQL string through the active dialect’s TranslateSQL(), enabling dialect-agnostic SQL assertions.
  • Fixtures: testdata/migrations/ holds real Fizz migration files that soda runs before the test suite starts (schema is set up via CI’s soda invocation, not in-process). testdata/models/ stores expected generated model output for code-generation tests. genny/model/_fixtures/ holds golden files for template output comparison.

Test patterns#

Table-driven tests#

  • Prevalence: Occasional — the grep for classic testCases := []struct / tt.Run patterns returns 0 hits, but a lighter table form appears in several unit tests (iterate over a slice of input values and assert in a loop). Examples:
    • finders_test.go:17 — tCases := []string{"Mark", "💩"} iterates over names to test Find with unicode.
    • connection_details_test.go — each Test_ConnectionDetails_Finalize_* function is effectively a separate table row, split into individual top-level functions rather than subtests. This is a common style choice in the gobuffalo ecosystem.
    • columns/columns_test.go:32–39 — for _, f := range []interface{}{foo{}, &foo{}} tests both pointer and value receivers in one loop.
  • Style: Inline slice of values rather than named struct with t.Run; the project predates the widespread adoption of t.Run-based subtests.

Mocking approach#

  • Strategy: Manual interface implementation in _test files (not a mock generator). The clearest example is genny/fizz/ctable/mocks_test.go, which hand-writes a mockTranslator struct implementing the full fizz.Translator interface (14 methods) with stub return values. This is used to test code generation templates without a real database translator.
  • No gomock/mockery: Zero references to gomock, mockery, or testify/mock. The project’s narrow dependency graph (few interfaces to mock) makes hand-written fakes feasible and avoids code generation complexity.

Integration tests#

  • Present: Yes — the majority of the test suite is integration tests.
  • How: Real databases via GitHub Actions service containers. Each CI job spins up one database engine (MySQL, PostgreSQL, CockroachDB, or SQLite) as a Docker service. The soda CLI is compiled first and used to drop/create/migrate the test schema. Then go test ./... runs with SODA_DIALECT set to the target engine.
    • SQLite runs on macOS, Windows, and Linux matrices simultaneously.
    • The sqlite build tag gates SQLite-specific code and tests via //go:build sqlite.
  • In-process skipping: All integration tests guard with if PDB == nil { t.Skip("skipping integration tests") } at the top, allowing go test ./... without SODA_DIALECT set to run only unit tests cleanly.
  • Transaction isolation: Every integration test body calls the transaction() helper, which uses PDB.Rollback() to wrap the test in a transaction that is always rolled back. This avoids test order dependencies without requiring fixture teardown.
  • Separation: No build tags separate integration from unit tests — the PDB == nil guard is the only gate. The root_integration_test.go file is explicitly named with _integration_ in the filename, which is the exception rather than the rule.

Benchmark tests#

  • Present: benchmarks_test.go contains 6 benchmarks comparing pop ORM vs raw sqlx calls for Create, Update, and Find, plus benchmarks for two alternative SQL ?→$N translation algorithms. These are useful for ORM overhead analysis and internal optimization decisions.

Dialect-specific tests#

  • dialect_sqlite_test.go (build-tagged //go:build sqlite) tests SQLite-specific URL parsing, memory-mode DB creation, and system table exclusion from schema dumps using t.TempDir() for isolated file-based databases.
  • dialect_nosqlite_test.go and connection_instrumented_nosqlite_test.go use //go:build !sqlite to guard tests that run only when SQLite is absent, preventing compilation errors on builds without the CGo dependency.
  • dialect_cockroach_test.go and dialect_mysql_test.go test dialect-specific connection string logic and SQL translation.

Testify suite usage#

  • pop_test.go defines PostgreSQLSuite, MySQLSuite, SQLiteSuite, and CockroachSuite as testify/suite.Suite embeds, but the suite methods are implemented in per-feature test files rather than on the suite types. The suites are dispatched by TestSpecificSuites which switches on SODA_DIALECT. This is an unusual hybrid: suites exist to group by dialect, but most tests are plain functions with a PDB == nil guard rather than suite methods.

Test quality observations#

  • What’s done well:

    • Transaction-scoped test isolation is elegant and robust — the Rollback() helper ensures zero cross-test contamination without teardown boilerplate. Any test that writes to the database starts with a guaranteed-empty state.
    • Real database testing across four engines (MySQL, PostgreSQL, CockroachDB, SQLite) on every PR gives high confidence in dialect compatibility. The CI matrix is comprehensive.
    • Fixture models in test package (pop_test.go) are rich and realistic — they exercise the full gamut of association types (has_many, has_one, belongs_to, many_to_many), custom primary keys, pointer fields, nullable types, embedded structs, and UUID IDs. These serve as living documentation of supported patterns.
    • Build-tag gating of CGo-dependent SQLite is clean — //go:build sqlite / //go:build !sqlite ensures non-CGo builds compile and test correctly, which is important for CI environments without a C toolchain.
    • Benchmarks alongside integration tests allow direct ORM-vs-raw comparison in a realistic environment.

  • What could improve:

    • No table-driven subtests with t.Run — the dialect connection string tests (Test_ConnectionDetails_Finalize_*) are split into many top-level functions rather than a single t.Run table. Consolidating them would improve readability and reduce boilerplate.
    • PDB == nil guard repeated in every integration test — the skip guard appears ~40+ times. A requireDB(t) helper or a TestMain that calls testing.Short() would centralize this.
    • No in-process database option — unit tests that need a real database must connect to an external engine. A SQLite in-memory database could allow many integration tests to run without external infrastructure, speeding up local development.
    • Suite types are underused — PostgreSQLSuite, MySQLSuite, etc. exist but almost no test methods are defined on them. The dialect switching is done via SODA_DIALECT env + PDB == nil guards, not suite membership. This creates an inconsistent mental model.
    • No context/cancellation tests — given that WithContext is a public API and contextStore is a core pattern, there are no tests verifying context cancellation propagates correctly to in-flight queries.

  • Patterns worth emulating:

    • transaction() / Rollback() pattern for test isolation — wrapping every test in a rolled-back transaction is a simple, zero-setup approach to database isolation that works across all SQL engines. Preferable to truncating tables or using separate schemas.
    • Rich fixture structs as documentation — defining realistic domain models in pop_test.go with all supported tag combinations makes the test file itself a reference for how to use the ORM.
    • Build-tag dialect gating — the //go:build sqlite / //go:build !sqlite duality is a clean pattern for optional CGo dependencies that other projects with similar concerns can directly adopt.
Backward Pop — API Surface PocketBase — Overview Forward