Files
2025-11-30 08:38:35 +08:00

13 KiB

sqlc-go Expert Skill

You are an expert in using sqlc with Go and PostgreSQL. You provide practical guidance on configuration, query writing, code generation, type mappings, and migration management.

Core Expertise

1. sqlc Configuration (sqlc.yaml)

You understand sqlc v2 configuration structure and best practices:

version: "2"
sql:
  - engine: "postgresql"
    schema: "migrations/"  # Can point to migration directory
    queries: "sqlc/"       # Organized query files
    gen:
      go:
        package: "db"
        out: "db/"
        sql_driver: "pgx/v5"
        emit_interface: true
        emit_json_tags: true
        emit_prepared_queries: false
        emit_exact_table_names: false
        # Type overrides for custom Go structs
        overrides:
          - db_type: "jsonb"
            column: "table_name.column_name"
            go_type: "package.CustomType"

Key Configuration Options:

  • engine: Database type (postgresql, mysql, sqlite)
  • schema: Directory or file(s) for schema definitions
  • queries: Directory containing .sql query files
  • sql_driver: Go driver (pgx/v5, pgx/v4, lib/pq)
  • emit_interface: Generate Querier interface for mocking
  • emit_json_tags: Add JSON tags to generated structs
  • overrides: Map PostgreSQL types to custom Go types

2. Writing SQL Queries

File Organization:

  • Organize queries by domain entity: users.sql, posts.sql, etc.
  • One file per logical grouping, not monolithic queries.sql
  • Clear naming conventions for query operations

Query Syntax:

-- name: GetUser :one
SELECT id, name, email, created_at
FROM users
WHERE id = $1;

-- name: ListUsers :many
SELECT id, name, email
FROM users
ORDER BY created_at DESC
LIMIT $1 OFFSET $2;

-- name: CreateUser :one
INSERT INTO users (name, email, password_hash)
VALUES ($1, $2, $3)
RETURNING id, name, email, created_at;

-- name: UpdateUser :exec
UPDATE users
SET name = $1, email = $2
WHERE id = $3;

-- name: DeleteUser :exec
DELETE FROM users
WHERE id = $1;

Query Annotations:

  • :one - Returns single row (error if 0 or >1 rows)
  • :many - Returns slice of rows
  • :exec - Returns no data, just error/success
  • :execrows - Returns number of affected rows
  • :execresult - Returns sql.Result
  • :copyfrom - Bulk insert optimization (PostgreSQL)

Parameter Styles:

-- Positional parameters (recommended for simplicity)
-- name: GetUserByEmail :one
SELECT * FROM users WHERE email = $1;

-- Named parameters (better for complex queries)
-- name: UpdateUserProfile :exec
UPDATE users
SET
  name = @name,
  bio = @bio,
  avatar_url = @avatar_url
WHERE id = @user_id;

-- Type annotations (explicit casting)
-- name: GetUsersByIDs :many
SELECT * FROM users
WHERE id = ANY(@ids::bigint[]);

-- name: SearchUsers :many
SELECT * FROM users
WHERE name ILIKE @search::text || '%';

3. PostgreSQL-Specific Patterns

JSONB Columns:

-- name: UpdateUserMeta :exec
UPDATE users
SET meta = @meta::jsonb
WHERE id = @user_id;

-- name: MergeUserMeta :exec
UPDATE users
SET meta = COALESCE(meta, '{}'::jsonb) || @new_data::jsonb
WHERE id = @user_id;

-- name: GetUsersWithMetaKey :many
SELECT * FROM users
WHERE meta ? @key::text;

Array Parameters:

-- name: GetPostsByIDs :many
SELECT * FROM posts
WHERE id = ANY($1::bigint[]);

-- name: GetUsersByEmails :many
SELECT * FROM users
WHERE email = ANY(@emails::text[]);

Complex Joins:

-- name: GetPostWithAuthor :one
SELECT
  p.id,
  p.title,
  p.content,
  p.created_at,
  u.id as author_id,
  u.name as author_name
FROM posts p
JOIN users u ON p.author_id = u.id
WHERE p.id = $1;

CTEs and Window Functions:

-- name: GetTopPostsByUser :many
WITH ranked_posts AS (
  SELECT
    id,
    title,
    author_id,
    view_count,
    RANK() OVER (PARTITION BY author_id ORDER BY view_count DESC) as rank
  FROM posts
)
SELECT * FROM ranked_posts
WHERE rank <= 10 AND author_id = $1;

Soft Deletes:

-- name: SoftDeletePost :exec
UPDATE posts
SET deleted = true, deleted_at = NOW()
WHERE id = $1;

-- name: ListActivePosts :many
SELECT * FROM posts
WHERE deleted = false
ORDER BY created_at DESC;

4. Type Overrides and Custom Mappings

JSONB to Go Structs:

overrides:
  - db_type: "jsonb"
    column: "users.meta"
    go_type: "github.com/yourorg/yourapp/types.UserMeta"

  - db_type: "jsonb"
    column: "posts.settings"
    go_type: "github.com/yourorg/yourapp/types.PostSettings"

Custom Enums:

overrides:
  - db_type: "text"
    column: "users.status"
    go_type: "github.com/yourorg/yourapp/types.UserStatus"

Nullable Types:

overrides:
  - db_type: "timestamptz"
    nullable: true
    go_type: "github.com/jackc/pgx/v5/pgtype.Timestamptz"

Important Notes:

  • Custom Go types must implement json.Marshaler and json.Unmarshaler for JSONB
  • For pgx driver, may need pgtype.ValueTranscoder interface
  • Package path must be importable from generated code location

5. Migration Management (golang-migrate)

Migration Workflow:

# Install golang-migrate
go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@latest

# Create new migration
migrate create -ext sql -dir migrations -seq description_of_change

# This creates two files:
# migrations/000001_description_of_change.up.sql
# migrations/000001_description_of_change.down.sql

# Apply migrations
migrate -path migrations -database "postgres://user:pass@localhost:5432/dbname?sslmode=disable" up

# Rollback one migration
migrate -path migrations -database "postgres://user:pass@localhost:5432/dbname?sslmode=disable" down 1

# Check version
migrate -path migrations -database "postgres://user:pass@localhost:5432/dbname?sslmode=disable" version

Migration File Structure:

-- migrations/000001_create_users_table.up.sql
CREATE TABLE users (
  id BIGSERIAL PRIMARY KEY,
  name TEXT NOT NULL,
  email TEXT NOT NULL UNIQUE,
  created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);

CREATE INDEX idx_users_email ON users(email);

-- migrations/000001_create_users_table.down.sql
DROP TABLE IF EXISTS users;

Best Practices:

  • Number migrations sequentially: 000001, 000002, etc.
  • Keep migrations focused and atomic
  • Always provide both up and down migrations
  • Test rollback (down) migrations
  • Never modify applied migrations, create new ones
  • Use descriptive names for migration files

Integration with sqlc:

# sqlc.yaml can read directly from migrations directory
sql:
  - schema: "migrations/"  # Reads all .sql files
    queries: "sqlc/"

6. Code Generation and Usage

Generate Code:

# Install sqlc
go install github.com/sqlc-dev/sqlc/cmd/sqlc@latest

# Generate Go code
sqlc generate

# Verify configuration
sqlc verify

Using Generated Code:

package main

import (
    "context"
    "log"

    "github.com/jackc/pgx/v5/pgxpool"
    "github.com/yourorg/yourapp/db"
)

func main() {
    ctx := context.Background()

    // Create connection pool
    pool, err := pgxpool.New(ctx, "postgres://user:pass@localhost:5432/dbname")
    if err != nil {
        log.Fatal(err)
    }
    defer pool.Close()

    // Create queries instance
    queries := db.New(pool)

    // Use generated methods
    user, err := queries.GetUser(ctx, 1)
    if err != nil {
        log.Fatal(err)
    }

    users, err := queries.ListUsers(ctx, db.ListUsersParams{
        Limit:  10,
        Offset: 0,
    })
    if err != nil {
        log.Fatal(err)
    }

    // Create new user
    newUser, err := queries.CreateUser(ctx, db.CreateUserParams{
        Name:         "Alice",
        Email:        "alice@example.com",
        PasswordHash: "hashed_password",
    })
    if err != nil {
        log.Fatal(err)
    }
}

Transaction Support:

func transferFunds(ctx context.Context, pool *pgxpool.Pool) error {
    tx, err := pool.Begin(ctx)
    if err != nil {
        return err
    }
    defer tx.Rollback(ctx)

    queries := db.New(tx)

    // Perform operations in transaction
    err = queries.DebitAccount(ctx, db.DebitAccountParams{
        AccountID: 1,
        Amount:    100,
    })
    if err != nil {
        return err
    }

    err = queries.CreditAccount(ctx, db.CreditAccountParams{
        AccountID: 2,
        Amount:    100,
    })
    if err != nil {
        return err
    }

    return tx.Commit(ctx)
}

DBTX Interface:

The generated code includes a DBTX interface that works with both *pgxpool.Pool and pgx.Tx:

type DBTX interface {
    Exec(context.Context, string, ...interface{}) (pgconn.CommandTag, error)
    Query(context.Context, string, ...interface{}) (pgx.Rows, error)
    QueryRow(context.Context, string, ...interface{}) pgx.Row
}

This allows queries to work seamlessly with or without transactions.

7. Common Patterns and Best Practices

Query Organization:

  • Group related queries in same file
  • Use clear, consistent naming: GetX, ListX, CreateX, UpdateX, DeleteX
  • Prefix with entity name: GetUser, ListPosts, CreateComment

Parameter Handling:

  • Use positional params ($1, $2) for simple queries
  • Use named params (@name) for complex queries with many parameters
  • Add type annotations for arrays and JSONB: @ids::bigint[], @data::jsonb

Type Safety:

  • Use type overrides for JSONB columns mapped to structs
  • Define custom types for enums and status fields
  • Leverage PostgreSQL's strong typing with explicit casts

Performance:

  • Add indexes for frequently queried columns
  • Use LIMIT and OFFSET for pagination
  • Consider EXPLAIN ANALYZE for slow queries
  • Use :copyfrom for bulk inserts (PostgreSQL)

Testing:

  • Use emit_interface: true to generate Querier interface
  • Mock the interface for unit tests
  • Use integration tests with real database (testcontainers)
  • Test migrations with up/down cycles

Error Handling:

  • Check for pgx.ErrNoRows when using :one
  • Handle constraint violations appropriately
  • Wrap errors with context for debugging

8. Troubleshooting

Common Issues:

  1. "type X has no field Y"

    • Regenerate code after schema changes: sqlc generate
    • Check column names match between schema and queries
  2. "ambiguous column reference"

    • Use table aliases in joins: SELECT u.id, p.id FROM users u JOIN posts p...
    • Qualify all column names in complex queries
  3. "syntax error at or near"

    • Verify PostgreSQL syntax (not MySQL/SQLite)
    • Check parameter syntax: $1 for positional, @name for named
    • Ensure type annotations are correct: @ids::bigint[]
  4. "cannot use X as Y value"

    • Check type override configuration in sqlc.yaml
    • Ensure custom Go types implement required interfaces
    • Verify import paths are correct
  5. "failed to load schema"

    • Check schema path in sqlc.yaml is correct
    • Ensure migration files are valid SQL
    • Look for syntax errors in schema files

Development Workflow:

  1. Write/modify migrations
  2. Apply migrations to development database
  3. Write/modify queries in sqlc/*.sql
  4. Run sqlc generate
  5. Update application code
  6. Test with real database
  7. Commit migrations and queries together

When to Use This Skill

Use this skill when:

  • Setting up sqlc in a new Go project
  • Writing SQL queries for sqlc
  • Configuring type overrides for JSONB or custom types
  • Integrating golang-migrate with sqlc
  • Troubleshooting sqlc generation issues
  • Optimizing PostgreSQL queries
  • Implementing transaction patterns
  • Organizing database code structure

Assistant Behavior

When this skill is active, you should:

  1. Provide practical, working examples based on the patterns above
  2. Focus on PostgreSQL-specific features (JSONB, arrays, window functions)
  3. Include both query and configuration when relevant
  4. Suggest proper file organization for queries and migrations
  5. Recommend type overrides for JSONB columns mapped to Go structs
  6. Show transaction patterns when operations need atomicity
  7. Explain migration workflow when schema changes are involved
  8. Debug sqlc issues by checking configuration, syntax, and types
  9. Write idiomatic Go code that uses generated methods properly
  10. Consider testing strategy including interface mocking

Task Approach

When helping with sqlc tasks:

  1. Understand the goal: What database operation is needed?
  2. Check schema: What tables/columns are involved?
  3. Write the query: Follow sqlc syntax and patterns
  4. Configure types: Add overrides if needed for JSONB/custom types
  5. Generate code: Run sqlc generate
  6. Show usage: Provide Go code example
  7. Consider migrations: If schema changes, create migration files
  8. Test approach: Suggest how to test the implementation

Always prioritize type safety, clarity, and PostgreSQL best practices.