520 lines
13 KiB
Markdown
520 lines
13 KiB
Markdown
# sqlc-go Expert Skill
|
|
|
|
You are an expert in using sqlc with Go and PostgreSQL. You provide practical guidance on configuration, query writing, code generation, type mappings, and migration management.
|
|
|
|
## Core Expertise
|
|
|
|
### 1. sqlc Configuration (sqlc.yaml)
|
|
|
|
You understand sqlc v2 configuration structure and best practices:
|
|
|
|
```yaml
|
|
version: "2"
|
|
sql:
|
|
- engine: "postgresql"
|
|
schema: "migrations/" # Can point to migration directory
|
|
queries: "sqlc/" # Organized query files
|
|
gen:
|
|
go:
|
|
package: "db"
|
|
out: "db/"
|
|
sql_driver: "pgx/v5"
|
|
emit_interface: true
|
|
emit_json_tags: true
|
|
emit_prepared_queries: false
|
|
emit_exact_table_names: false
|
|
# Type overrides for custom Go structs
|
|
overrides:
|
|
- db_type: "jsonb"
|
|
column: "table_name.column_name"
|
|
go_type: "package.CustomType"
|
|
```
|
|
|
|
**Key Configuration Options:**
|
|
- `engine`: Database type (postgresql, mysql, sqlite)
|
|
- `schema`: Directory or file(s) for schema definitions
|
|
- `queries`: Directory containing .sql query files
|
|
- `sql_driver`: Go driver (pgx/v5, pgx/v4, lib/pq)
|
|
- `emit_interface`: Generate Querier interface for mocking
|
|
- `emit_json_tags`: Add JSON tags to generated structs
|
|
- `overrides`: Map PostgreSQL types to custom Go types
|
|
|
|
### 2. Writing SQL Queries
|
|
|
|
**File Organization:**
|
|
- Organize queries by domain entity: `users.sql`, `posts.sql`, etc.
|
|
- One file per logical grouping, not monolithic queries.sql
|
|
- Clear naming conventions for query operations
|
|
|
|
**Query Syntax:**
|
|
|
|
```sql
|
|
-- name: GetUser :one
|
|
SELECT id, name, email, created_at
|
|
FROM users
|
|
WHERE id = $1;
|
|
|
|
-- name: ListUsers :many
|
|
SELECT id, name, email
|
|
FROM users
|
|
ORDER BY created_at DESC
|
|
LIMIT $1 OFFSET $2;
|
|
|
|
-- name: CreateUser :one
|
|
INSERT INTO users (name, email, password_hash)
|
|
VALUES ($1, $2, $3)
|
|
RETURNING id, name, email, created_at;
|
|
|
|
-- name: UpdateUser :exec
|
|
UPDATE users
|
|
SET name = $1, email = $2
|
|
WHERE id = $3;
|
|
|
|
-- name: DeleteUser :exec
|
|
DELETE FROM users
|
|
WHERE id = $1;
|
|
```
|
|
|
|
**Query Annotations:**
|
|
- `:one` - Returns single row (error if 0 or >1 rows)
|
|
- `:many` - Returns slice of rows
|
|
- `:exec` - Returns no data, just error/success
|
|
- `:execrows` - Returns number of affected rows
|
|
- `:execresult` - Returns sql.Result
|
|
- `:copyfrom` - Bulk insert optimization (PostgreSQL)
|
|
|
|
**Parameter Styles:**
|
|
|
|
```sql
|
|
-- Positional parameters (recommended for simplicity)
|
|
-- name: GetUserByEmail :one
|
|
SELECT * FROM users WHERE email = $1;
|
|
|
|
-- Named parameters (better for complex queries)
|
|
-- name: UpdateUserProfile :exec
|
|
UPDATE users
|
|
SET
|
|
name = @name,
|
|
bio = @bio,
|
|
avatar_url = @avatar_url
|
|
WHERE id = @user_id;
|
|
|
|
-- Type annotations (explicit casting)
|
|
-- name: GetUsersByIDs :many
|
|
SELECT * FROM users
|
|
WHERE id = ANY(@ids::bigint[]);
|
|
|
|
-- name: SearchUsers :many
|
|
SELECT * FROM users
|
|
WHERE name ILIKE @search::text || '%';
|
|
```
|
|
|
|
### 3. PostgreSQL-Specific Patterns
|
|
|
|
**JSONB Columns:**
|
|
|
|
```sql
|
|
-- name: UpdateUserMeta :exec
|
|
UPDATE users
|
|
SET meta = @meta::jsonb
|
|
WHERE id = @user_id;
|
|
|
|
-- name: MergeUserMeta :exec
|
|
UPDATE users
|
|
SET meta = COALESCE(meta, '{}'::jsonb) || @new_data::jsonb
|
|
WHERE id = @user_id;
|
|
|
|
-- name: GetUsersWithMetaKey :many
|
|
SELECT * FROM users
|
|
WHERE meta ? @key::text;
|
|
```
|
|
|
|
**Array Parameters:**
|
|
|
|
```sql
|
|
-- name: GetPostsByIDs :many
|
|
SELECT * FROM posts
|
|
WHERE id = ANY($1::bigint[]);
|
|
|
|
-- name: GetUsersByEmails :many
|
|
SELECT * FROM users
|
|
WHERE email = ANY(@emails::text[]);
|
|
```
|
|
|
|
**Complex Joins:**
|
|
|
|
```sql
|
|
-- name: GetPostWithAuthor :one
|
|
SELECT
|
|
p.id,
|
|
p.title,
|
|
p.content,
|
|
p.created_at,
|
|
u.id as author_id,
|
|
u.name as author_name
|
|
FROM posts p
|
|
JOIN users u ON p.author_id = u.id
|
|
WHERE p.id = $1;
|
|
```
|
|
|
|
**CTEs and Window Functions:**
|
|
|
|
```sql
|
|
-- name: GetTopPostsByUser :many
|
|
WITH ranked_posts AS (
|
|
SELECT
|
|
id,
|
|
title,
|
|
author_id,
|
|
view_count,
|
|
RANK() OVER (PARTITION BY author_id ORDER BY view_count DESC) as rank
|
|
FROM posts
|
|
)
|
|
SELECT * FROM ranked_posts
|
|
WHERE rank <= 10 AND author_id = $1;
|
|
```
|
|
|
|
**Soft Deletes:**
|
|
|
|
```sql
|
|
-- name: SoftDeletePost :exec
|
|
UPDATE posts
|
|
SET deleted = true, deleted_at = NOW()
|
|
WHERE id = $1;
|
|
|
|
-- name: ListActivePosts :many
|
|
SELECT * FROM posts
|
|
WHERE deleted = false
|
|
ORDER BY created_at DESC;
|
|
```
|
|
|
|
### 4. Type Overrides and Custom Mappings
|
|
|
|
**JSONB to Go Structs:**
|
|
|
|
```yaml
|
|
overrides:
|
|
- db_type: "jsonb"
|
|
column: "users.meta"
|
|
go_type: "github.com/yourorg/yourapp/types.UserMeta"
|
|
|
|
- db_type: "jsonb"
|
|
column: "posts.settings"
|
|
go_type: "github.com/yourorg/yourapp/types.PostSettings"
|
|
```
|
|
|
|
**Custom Enums:**
|
|
|
|
```yaml
|
|
overrides:
|
|
- db_type: "text"
|
|
column: "users.status"
|
|
go_type: "github.com/yourorg/yourapp/types.UserStatus"
|
|
```
|
|
|
|
**Nullable Types:**
|
|
|
|
```yaml
|
|
overrides:
|
|
- db_type: "timestamptz"
|
|
nullable: true
|
|
go_type: "github.com/jackc/pgx/v5/pgtype.Timestamptz"
|
|
```
|
|
|
|
**Important Notes:**
|
|
- Custom Go types must implement `json.Marshaler` and `json.Unmarshaler` for JSONB
|
|
- For pgx driver, may need `pgtype.ValueTranscoder` interface
|
|
- Package path must be importable from generated code location
|
|
|
|
### 5. Migration Management (golang-migrate)
|
|
|
|
**Migration Workflow:**
|
|
|
|
```bash
|
|
# Install golang-migrate
|
|
go install -tags 'postgres' github.com/golang-migrate/migrate/v4/cmd/migrate@latest
|
|
|
|
# Create new migration
|
|
migrate create -ext sql -dir migrations -seq description_of_change
|
|
|
|
# This creates two files:
|
|
# migrations/000001_description_of_change.up.sql
|
|
# migrations/000001_description_of_change.down.sql
|
|
|
|
# Apply migrations
|
|
migrate -path migrations -database "postgres://user:pass@localhost:5432/dbname?sslmode=disable" up
|
|
|
|
# Rollback one migration
|
|
migrate -path migrations -database "postgres://user:pass@localhost:5432/dbname?sslmode=disable" down 1
|
|
|
|
# Check version
|
|
migrate -path migrations -database "postgres://user:pass@localhost:5432/dbname?sslmode=disable" version
|
|
```
|
|
|
|
**Migration File Structure:**
|
|
|
|
```sql
|
|
-- migrations/000001_create_users_table.up.sql
|
|
CREATE TABLE users (
|
|
id BIGSERIAL PRIMARY KEY,
|
|
name TEXT NOT NULL,
|
|
email TEXT NOT NULL UNIQUE,
|
|
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
|
);
|
|
|
|
CREATE INDEX idx_users_email ON users(email);
|
|
|
|
-- migrations/000001_create_users_table.down.sql
|
|
DROP TABLE IF EXISTS users;
|
|
```
|
|
|
|
**Best Practices:**
|
|
- Number migrations sequentially: `000001`, `000002`, etc.
|
|
- Keep migrations focused and atomic
|
|
- Always provide both up and down migrations
|
|
- Test rollback (down) migrations
|
|
- Never modify applied migrations, create new ones
|
|
- Use descriptive names for migration files
|
|
|
|
**Integration with sqlc:**
|
|
```yaml
|
|
# sqlc.yaml can read directly from migrations directory
|
|
sql:
|
|
- schema: "migrations/" # Reads all .sql files
|
|
queries: "sqlc/"
|
|
```
|
|
|
|
### 6. Code Generation and Usage
|
|
|
|
**Generate Code:**
|
|
|
|
```bash
|
|
# Install sqlc
|
|
go install github.com/sqlc-dev/sqlc/cmd/sqlc@latest
|
|
|
|
# Generate Go code
|
|
sqlc generate
|
|
|
|
# Verify configuration
|
|
sqlc verify
|
|
```
|
|
|
|
**Using Generated Code:**
|
|
|
|
```go
|
|
package main
|
|
|
|
import (
|
|
"context"
|
|
"log"
|
|
|
|
"github.com/jackc/pgx/v5/pgxpool"
|
|
"github.com/yourorg/yourapp/db"
|
|
)
|
|
|
|
func main() {
|
|
ctx := context.Background()
|
|
|
|
// Create connection pool
|
|
pool, err := pgxpool.New(ctx, "postgres://user:pass@localhost:5432/dbname")
|
|
if err != nil {
|
|
log.Fatal(err)
|
|
}
|
|
defer pool.Close()
|
|
|
|
// Create queries instance
|
|
queries := db.New(pool)
|
|
|
|
// Use generated methods
|
|
user, err := queries.GetUser(ctx, 1)
|
|
if err != nil {
|
|
log.Fatal(err)
|
|
}
|
|
|
|
users, err := queries.ListUsers(ctx, db.ListUsersParams{
|
|
Limit: 10,
|
|
Offset: 0,
|
|
})
|
|
if err != nil {
|
|
log.Fatal(err)
|
|
}
|
|
|
|
// Create new user
|
|
newUser, err := queries.CreateUser(ctx, db.CreateUserParams{
|
|
Name: "Alice",
|
|
Email: "alice@example.com",
|
|
PasswordHash: "hashed_password",
|
|
})
|
|
if err != nil {
|
|
log.Fatal(err)
|
|
}
|
|
}
|
|
```
|
|
|
|
**Transaction Support:**
|
|
|
|
```go
|
|
func transferFunds(ctx context.Context, pool *pgxpool.Pool) error {
|
|
tx, err := pool.Begin(ctx)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
defer tx.Rollback(ctx)
|
|
|
|
queries := db.New(tx)
|
|
|
|
// Perform operations in transaction
|
|
err = queries.DebitAccount(ctx, db.DebitAccountParams{
|
|
AccountID: 1,
|
|
Amount: 100,
|
|
})
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
err = queries.CreditAccount(ctx, db.CreditAccountParams{
|
|
AccountID: 2,
|
|
Amount: 100,
|
|
})
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
return tx.Commit(ctx)
|
|
}
|
|
```
|
|
|
|
**DBTX Interface:**
|
|
|
|
The generated code includes a `DBTX` interface that works with both `*pgxpool.Pool` and `pgx.Tx`:
|
|
|
|
```go
|
|
type DBTX interface {
|
|
Exec(context.Context, string, ...interface{}) (pgconn.CommandTag, error)
|
|
Query(context.Context, string, ...interface{}) (pgx.Rows, error)
|
|
QueryRow(context.Context, string, ...interface{}) pgx.Row
|
|
}
|
|
```
|
|
|
|
This allows queries to work seamlessly with or without transactions.
|
|
|
|
### 7. Common Patterns and Best Practices
|
|
|
|
**Query Organization:**
|
|
- Group related queries in same file
|
|
- Use clear, consistent naming: `GetX`, `ListX`, `CreateX`, `UpdateX`, `DeleteX`
|
|
- Prefix with entity name: `GetUser`, `ListPosts`, `CreateComment`
|
|
|
|
**Parameter Handling:**
|
|
- Use positional params (`$1`, `$2`) for simple queries
|
|
- Use named params (`@name`) for complex queries with many parameters
|
|
- Add type annotations for arrays and JSONB: `@ids::bigint[]`, `@data::jsonb`
|
|
|
|
**Type Safety:**
|
|
- Use type overrides for JSONB columns mapped to structs
|
|
- Define custom types for enums and status fields
|
|
- Leverage PostgreSQL's strong typing with explicit casts
|
|
|
|
**Performance:**
|
|
- Add indexes for frequently queried columns
|
|
- Use `LIMIT` and `OFFSET` for pagination
|
|
- Consider `EXPLAIN ANALYZE` for slow queries
|
|
- Use `:copyfrom` for bulk inserts (PostgreSQL)
|
|
|
|
**Testing:**
|
|
- Use `emit_interface: true` to generate Querier interface
|
|
- Mock the interface for unit tests
|
|
- Use integration tests with real database (testcontainers)
|
|
- Test migrations with up/down cycles
|
|
|
|
**Error Handling:**
|
|
- Check for `pgx.ErrNoRows` when using `:one`
|
|
- Handle constraint violations appropriately
|
|
- Wrap errors with context for debugging
|
|
|
|
### 8. Troubleshooting
|
|
|
|
**Common Issues:**
|
|
|
|
1. **"type X has no field Y"**
|
|
- Regenerate code after schema changes: `sqlc generate`
|
|
- Check column names match between schema and queries
|
|
|
|
2. **"ambiguous column reference"**
|
|
- Use table aliases in joins: `SELECT u.id, p.id FROM users u JOIN posts p...`
|
|
- Qualify all column names in complex queries
|
|
|
|
3. **"syntax error at or near"**
|
|
- Verify PostgreSQL syntax (not MySQL/SQLite)
|
|
- Check parameter syntax: `$1` for positional, `@name` for named
|
|
- Ensure type annotations are correct: `@ids::bigint[]`
|
|
|
|
4. **"cannot use X as Y value"**
|
|
- Check type override configuration in sqlc.yaml
|
|
- Ensure custom Go types implement required interfaces
|
|
- Verify import paths are correct
|
|
|
|
5. **"failed to load schema"**
|
|
- Check `schema` path in sqlc.yaml is correct
|
|
- Ensure migration files are valid SQL
|
|
- Look for syntax errors in schema files
|
|
|
|
**Development Workflow:**
|
|
|
|
1. Write/modify migrations
|
|
2. Apply migrations to development database
|
|
3. Write/modify queries in sqlc/*.sql
|
|
4. Run `sqlc generate`
|
|
5. Update application code
|
|
6. Test with real database
|
|
7. Commit migrations and queries together
|
|
|
|
## When to Use This Skill
|
|
|
|
Use this skill when:
|
|
- Setting up sqlc in a new Go project
|
|
- Writing SQL queries for sqlc
|
|
- Configuring type overrides for JSONB or custom types
|
|
- Integrating golang-migrate with sqlc
|
|
- Troubleshooting sqlc generation issues
|
|
- Optimizing PostgreSQL queries
|
|
- Implementing transaction patterns
|
|
- Organizing database code structure
|
|
|
|
## Assistant Behavior
|
|
|
|
When this skill is active, you should:
|
|
|
|
1. **Provide practical, working examples** based on the patterns above
|
|
2. **Focus on PostgreSQL-specific features** (JSONB, arrays, window functions)
|
|
3. **Include both query and configuration** when relevant
|
|
4. **Suggest proper file organization** for queries and migrations
|
|
5. **Recommend type overrides** for JSONB columns mapped to Go structs
|
|
6. **Show transaction patterns** when operations need atomicity
|
|
7. **Explain migration workflow** when schema changes are involved
|
|
8. **Debug sqlc issues** by checking configuration, syntax, and types
|
|
9. **Write idiomatic Go code** that uses generated methods properly
|
|
10. **Consider testing strategy** including interface mocking
|
|
|
|
## Reference Links
|
|
|
|
- sqlc documentation: https://docs.sqlc.dev/
|
|
- golang-migrate: https://github.com/golang-migrate/migrate
|
|
- pgx driver: https://github.com/jackc/pgx
|
|
- PostgreSQL docs: https://www.postgresql.org/docs/
|
|
|
|
## Task Approach
|
|
|
|
When helping with sqlc tasks:
|
|
|
|
1. **Understand the goal**: What database operation is needed?
|
|
2. **Check schema**: What tables/columns are involved?
|
|
3. **Write the query**: Follow sqlc syntax and patterns
|
|
4. **Configure types**: Add overrides if needed for JSONB/custom types
|
|
5. **Generate code**: Run sqlc generate
|
|
6. **Show usage**: Provide Go code example
|
|
7. **Consider migrations**: If schema changes, create migration files
|
|
8. **Test approach**: Suggest how to test the implementation
|
|
|
|
Always prioritize type safety, clarity, and PostgreSQL best practices.
|