Initial commit
This commit is contained in:
Zhongwei Li
271
skills/optimizing-query-selection/SKILL.md
Normal file
271
skills/optimizing-query-selection/SKILL.md
Normal file
@@ -0,0 +1,271 @@
|
||||
---
|
||||
name: optimizing-query-selection
|
||||
description: Optimize queries by selecting only required fields and avoiding N+1 problems. Use when writing queries with relations or large result sets.
|
||||
allowed-tools: Read, Write, Edit
|
||||
version: 1.0.0
|
||||
---
|
||||
|
||||
# Query Select Optimization
|
||||
|
||||
Optimize Prisma 6 queries through selective field loading and relation batching to prevent N+1 problems and reduce data transfer.
|
||||
|
||||
---
|
||||
|
||||
<role>
|
||||
Optimize Prisma 6 queries by selecting required fields only, properly loading relations to prevent N+1 problems while minimizing data transfer and memory usage.
|
||||
</role>
|
||||
|
||||
<when-to-activate>
|
||||
- Writing user-facing data queries
|
||||
- Loading models with relations
|
||||
- Building API endpoints or GraphQL resolvers
|
||||
- Optimizing slow queries; reducing database load
|
||||
- Working with large result sets
|
||||
</when-to-activate>
|
||||
|
||||
<workflow>
|
||||
## Optimization Workflow
|
||||
|
||||
1. **Identify:** Determine required fields, relations to load, relation count needs, full vs. specific fields
|
||||
2. **Choose:** `include` (prototyping, most fields) vs. `select` (production, API responses, performance-critical)
|
||||
3. **Implement:** Use `select` for precise control, nest relations with `select`, use `_count` instead of loading all records, limit relation results with `take`
|
||||
4. **Index:** Fields in `where` clauses, `orderBy` fields, composite indexes for filtered relations
|
||||
5. **Validate:** Enable query logging for single-query verification, test with realistic data volumes, measure payload size and query duration
|
||||
</workflow>
|
||||
|
||||
<core-principles>
|
||||
## Core Principles
|
||||
|
||||
### 1. Select Only Required Fields
|
||||
|
||||
**Problem:** Fetching entire models wastes bandwidth and memory
|
||||
|
||||
```typescript
|
||||
const users = await prisma.user.findMany()
|
||||
```
|
||||
|
||||
**Solution:** Use `select` to fetch only needed fields
|
||||
|
||||
```typescript
|
||||
const users = await prisma.user.findMany({
|
||||
select: {
|
||||
id: true,
|
||||
email: true,
|
||||
name: true,
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
**Performance Impact:**
|
||||
- Reduces data transfer by 60-90% for models with many fields
|
||||
- Faster JSON serialization
|
||||
- Lower memory usage
|
||||
- Excludes sensitive fields by default
|
||||
|
||||
### 2. Include vs Select
|
||||
|
||||
**Include:** Adds relations to full model
|
||||
|
||||
```typescript
|
||||
const user = await prisma.user.findUnique({
|
||||
where: { id: 1 },
|
||||
include: {
|
||||
posts: true,
|
||||
profile: true,
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
**Select:** Precise control over all fields
|
||||
|
||||
```typescript
|
||||
const user = await prisma.user.findUnique({
|
||||
where: { id: 1 },
|
||||
select: {
|
||||
id: true,
|
||||
email: true,
|
||||
posts: {
|
||||
select: {
|
||||
id: true,
|
||||
title: true,
|
||||
published: true,
|
||||
},
|
||||
},
|
||||
profile: {
|
||||
select: {
|
||||
bio: true,
|
||||
avatar: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
**When to Use:**
|
||||
- `include`: Quick prototyping, need most fields
|
||||
- `select`: Production code, API responses, performance-critical paths
|
||||
|
||||
### 3. Preventing N+1 Queries
|
||||
|
||||
**N+1 Problem:** Separate query for each relation
|
||||
|
||||
```typescript
|
||||
const posts = await prisma.post.findMany()
|
||||
|
||||
for (const post of posts) {
|
||||
const author = await prisma.user.findUnique({
|
||||
where: { id: post.authorId },
|
||||
})
|
||||
}
|
||||
```
|
||||
|
||||
**Solution:** Use `include` or `select` with relations
|
||||
|
||||
```typescript
|
||||
const posts = await prisma.post.findMany({
|
||||
include: {
|
||||
author: true,
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
**Better:** Select only needed author fields
|
||||
|
||||
```typescript
|
||||
const posts = await prisma.post.findMany({
|
||||
select: {
|
||||
id: true,
|
||||
title: true,
|
||||
content: true,
|
||||
author: {
|
||||
select: {
|
||||
id: true,
|
||||
name: true,
|
||||
email: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
### 4. Relation Counting
|
||||
|
||||
**Problem:** Loading all relations just to count them
|
||||
|
||||
```typescript
|
||||
const user = await prisma.user.findUnique({
|
||||
where: { id: 1 },
|
||||
include: {
|
||||
posts: true,
|
||||
},
|
||||
})
|
||||
|
||||
const postCount = user.posts.length
|
||||
```
|
||||
|
||||
**Solution:** Use `_count` for efficient aggregation
|
||||
|
||||
```typescript
|
||||
const user = await prisma.user.findUnique({
|
||||
where: { id: 1 },
|
||||
select: {
|
||||
id: true,
|
||||
name: true,
|
||||
_count: {
|
||||
select: {
|
||||
posts: true,
|
||||
comments: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
**Result:**
|
||||
```typescript
|
||||
{
|
||||
id: 1,
|
||||
name: "Alice",
|
||||
_count: {
|
||||
posts: 42,
|
||||
comments: 128
|
||||
}
|
||||
}
|
||||
```
|
||||
</core-principles>
|
||||
|
||||
<quick-reference>
|
||||
## Quick Reference
|
||||
|
||||
### Optimized Query Pattern
|
||||
|
||||
```typescript
|
||||
const optimized = await prisma.model.findMany({
|
||||
where: {},
|
||||
select: {
|
||||
field1: true,
|
||||
field2: true,
|
||||
relation: {
|
||||
select: {
|
||||
field: true,
|
||||
},
|
||||
take: 10,
|
||||
},
|
||||
_count: {
|
||||
select: {
|
||||
relation: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
orderBy: { field: 'desc' },
|
||||
take: 20,
|
||||
skip: 0,
|
||||
})
|
||||
```
|
||||
|
||||
### Key Takeaways
|
||||
|
||||
- Default to `select` for all production queries
|
||||
- Use `include` only for prototyping
|
||||
- Always use `_count` for counting relations
|
||||
- Combine selection with filtering and pagination
|
||||
- Prevent N+1 by loading relations upfront
|
||||
- Select minimal fields for list views, more for detail views
|
||||
</quick-reference>
|
||||
|
||||
<constraints>
|
||||
## Constraints and Guidelines
|
||||
|
||||
**MUST:**
|
||||
- Use `select` for all API responses
|
||||
- Load relations in same query (prevent N+1)
|
||||
- Use `_count` for relation counts
|
||||
- Add indexes for filtered/ordered fields
|
||||
- Test with realistic data volumes
|
||||
|
||||
**SHOULD:**
|
||||
- Limit relation results with `take`
|
||||
- Create reusable selection objects
|
||||
- Enable query logging during development
|
||||
- Measure performance improvements
|
||||
- Document selection patterns
|
||||
|
||||
**NEVER:**
|
||||
- Use `include` in production without field selection
|
||||
- Load relations in loops (N+1)
|
||||
- Fetch full models when only counts needed
|
||||
- Over-fetch nested relations
|
||||
- Skip indexes on commonly queried fields
|
||||
</constraints>
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
For detailed patterns and examples, see:
|
||||
|
||||
- [Nested Selection Patterns](./references/nested-selection.md) - Deep relation hierarchies and complex selections
|
||||
- [API Optimization Patterns](./references/api-optimization.md) - List vs detail views, pagination with select
|
||||
- [N+1 Prevention Guide](./references/n-plus-one-prevention.md) - Detailed anti-patterns and solutions
|
||||
- [Type Safety Guide](./references/type-safety.md) - TypeScript types and reusable selection objects
|
||||
- [Performance Verification](./references/performance-verification.md) - Testing and validation techniques
|
||||
139
skills/optimizing-query-selection/references/api-optimization.md
Normal file
139
skills/optimizing-query-selection/references/api-optimization.md
Normal file
@@ -0,0 +1,139 @@
|
||||
# API Optimization Patterns
|
||||
|
||||
## API Endpoint Optimization
|
||||
|
||||
```typescript
|
||||
export async function GET(request: Request) {
|
||||
const posts = await prisma.post.findMany({
|
||||
where: { published: true },
|
||||
select: {
|
||||
id: true,
|
||||
title: true,
|
||||
slug: true,
|
||||
excerpt: true,
|
||||
publishedAt: true,
|
||||
author: {
|
||||
select: {
|
||||
name: true,
|
||||
avatar: true,
|
||||
},
|
||||
},
|
||||
_count: {
|
||||
select: {
|
||||
comments: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
orderBy: {
|
||||
publishedAt: 'desc',
|
||||
},
|
||||
take: 20,
|
||||
})
|
||||
|
||||
return Response.json(posts)
|
||||
}
|
||||
```
|
||||
|
||||
## List vs Detail Views
|
||||
|
||||
### List View: Minimal Fields
|
||||
|
||||
```typescript
|
||||
const users = await prisma.user.findMany({
|
||||
select: {
|
||||
id: true,
|
||||
name: true,
|
||||
email: true,
|
||||
role: true,
|
||||
_count: {
|
||||
select: {
|
||||
posts: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
### Detail View: More Complete Data
|
||||
|
||||
```typescript
|
||||
const user = await prisma.user.findUnique({
|
||||
where: { id },
|
||||
select: {
|
||||
id: true,
|
||||
name: true,
|
||||
email: true,
|
||||
role: true,
|
||||
bio: true,
|
||||
avatar: true,
|
||||
createdAt: true,
|
||||
posts: {
|
||||
select: {
|
||||
id: true,
|
||||
title: true,
|
||||
publishedAt: true,
|
||||
_count: {
|
||||
select: {
|
||||
comments: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
orderBy: {
|
||||
publishedAt: 'desc',
|
||||
},
|
||||
take: 10,
|
||||
},
|
||||
_count: {
|
||||
select: {
|
||||
posts: true,
|
||||
comments: true,
|
||||
followers: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
## Pagination with Select
|
||||
|
||||
```typescript
|
||||
async function getPaginatedPosts(page: number, pageSize: number) {
|
||||
const [posts, total] = await Promise.all([
|
||||
prisma.post.findMany({
|
||||
select: {
|
||||
id: true,
|
||||
title: true,
|
||||
excerpt: true,
|
||||
author: {
|
||||
select: {
|
||||
name: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
skip: page * pageSize,
|
||||
take: pageSize,
|
||||
orderBy: {
|
||||
createdAt: 'desc',
|
||||
},
|
||||
}),
|
||||
prisma.post.count(),
|
||||
])
|
||||
|
||||
return {
|
||||
posts,
|
||||
pagination: {
|
||||
page,
|
||||
pageSize,
|
||||
total,
|
||||
pages: Math.ceil(total / pageSize),
|
||||
},
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Key Patterns
|
||||
|
||||
- **List views:** Minimize fields, use `_count` for relations
|
||||
- **Detail views:** Include necessary relations with limits
|
||||
- **API responses:** Always use `select` to control shape
|
||||
- **Pagination:** Combine `select` with `take`/`skip`
|
||||
@@ -0,0 +1,112 @@
|
||||
# N+1 Prevention Guide
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
### Over-fetching
|
||||
|
||||
**Problem:**
|
||||
```typescript
|
||||
const user = await prisma.user.findUnique({
|
||||
where: { id },
|
||||
include: {
|
||||
posts: {
|
||||
include: {
|
||||
comments: {
|
||||
include: {
|
||||
author: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
**Issue:** Fetches thousands of records, massive data transfer
|
||||
|
||||
**Fix:** Use select with limits
|
||||
|
||||
```typescript
|
||||
const user = await prisma.user.findUnique({
|
||||
where: { id },
|
||||
select: {
|
||||
id: true,
|
||||
name: true,
|
||||
posts: {
|
||||
select: {
|
||||
id: true,
|
||||
title: true,
|
||||
_count: {
|
||||
select: {
|
||||
comments: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
take: 10,
|
||||
orderBy: {
|
||||
createdAt: 'desc',
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
### Inconsistent Selection
|
||||
|
||||
**Problem:**
|
||||
```typescript
|
||||
const posts = await prisma.post.findMany({
|
||||
include: {
|
||||
author: true,
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
**Issue:** Full author object when only name needed
|
||||
|
||||
**Fix:** Select specific fields
|
||||
|
||||
```typescript
|
||||
const posts = await prisma.post.findMany({
|
||||
select: {
|
||||
id: true,
|
||||
title: true,
|
||||
author: {
|
||||
select: {
|
||||
name: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
### Selecting Then Filtering
|
||||
|
||||
**Problem:**
|
||||
```typescript
|
||||
const users = await prisma.user.findMany()
|
||||
const activeUsers = users.filter(u => u.status === 'active')
|
||||
```
|
||||
|
||||
**Issue:** Fetches all users, filters in application
|
||||
|
||||
**Fix:** Filter in database
|
||||
|
||||
```typescript
|
||||
const activeUsers = await prisma.user.findMany({
|
||||
where: { status: 'active' },
|
||||
select: {
|
||||
id: true,
|
||||
name: true,
|
||||
email: true,
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
## Prevention Strategies
|
||||
|
||||
1. **Always load relations upfront** - Never query in loops
|
||||
2. **Use select with relations** - Don't fetch unnecessary fields
|
||||
3. **Add take limits** - Prevent accidental bulk loads
|
||||
4. **Use _count** - Don't load relations just to count
|
||||
5. **Test with realistic data** - N+1 only shows at scale
|
||||
@@ -0,0 +1,81 @@
|
||||
# Nested Selection Patterns
|
||||
|
||||
## Deep Relation Hierarchies
|
||||
|
||||
Select fields deep in relation hierarchies:
|
||||
|
||||
```typescript
|
||||
const posts = await prisma.post.findMany({
|
||||
select: {
|
||||
title: true,
|
||||
author: {
|
||||
select: {
|
||||
name: true,
|
||||
profile: {
|
||||
select: {
|
||||
avatar: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
},
|
||||
comments: {
|
||||
select: {
|
||||
content: true,
|
||||
author: {
|
||||
select: {
|
||||
name: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
take: 5,
|
||||
orderBy: {
|
||||
createdAt: 'desc',
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
## Combining Select with Filtering
|
||||
|
||||
Optimize both data transfer and query performance:
|
||||
|
||||
```typescript
|
||||
const recentPosts = await prisma.post.findMany({
|
||||
where: {
|
||||
published: true,
|
||||
createdAt: {
|
||||
gte: new Date(Date.now() - 7 * 24 * 60 * 60 * 1000),
|
||||
},
|
||||
},
|
||||
select: {
|
||||
id: true,
|
||||
title: true,
|
||||
excerpt: true,
|
||||
createdAt: true,
|
||||
author: {
|
||||
select: {
|
||||
id: true,
|
||||
name: true,
|
||||
},
|
||||
},
|
||||
_count: {
|
||||
select: {
|
||||
comments: true,
|
||||
likes: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
orderBy: {
|
||||
createdAt: 'desc',
|
||||
},
|
||||
take: 10,
|
||||
})
|
||||
```
|
||||
|
||||
## Key Principles
|
||||
|
||||
- Nest selections to match data shape requirements
|
||||
- Use `take` on nested relations to prevent over-fetching
|
||||
- Combine `orderBy` with nested relations for sorted results
|
||||
- Use `_count` for relation counts instead of loading all records
|
||||
@@ -0,0 +1,101 @@
|
||||
# Performance Verification
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
After optimization, verify improvements:
|
||||
|
||||
1. **Data Size:** Check response payload size
|
||||
2. **Query Time:** Measure database query duration
|
||||
3. **Query Count:** Ensure single query instead of N+1
|
||||
4. **Memory Usage:** Monitor application memory
|
||||
|
||||
## Enable Query Logging
|
||||
|
||||
```typescript
|
||||
const prisma = new PrismaClient({
|
||||
log: [
|
||||
{ emit: 'event', level: 'query' },
|
||||
],
|
||||
})
|
||||
|
||||
prisma.$on('query', (e) => {
|
||||
console.log('Query: ' + e.query)
|
||||
console.log('Duration: ' + e.duration + 'ms')
|
||||
})
|
||||
```
|
||||
|
||||
## Performance Testing
|
||||
|
||||
```typescript
|
||||
async function testQueryPerformance() {
|
||||
console.time('Unoptimized')
|
||||
await prisma.user.findMany({
|
||||
include: { posts: true }
|
||||
})
|
||||
console.timeEnd('Unoptimized')
|
||||
|
||||
console.time('Optimized')
|
||||
await prisma.user.findMany({
|
||||
select: {
|
||||
id: true,
|
||||
name: true,
|
||||
_count: { select: { posts: true } }
|
||||
}
|
||||
})
|
||||
console.timeEnd('Optimized')
|
||||
}
|
||||
```
|
||||
|
||||
## Payload Size Comparison
|
||||
|
||||
```typescript
|
||||
async function comparePayloadSize() {
|
||||
const full = await prisma.post.findMany()
|
||||
const optimized = await prisma.post.findMany({
|
||||
select: {
|
||||
id: true,
|
||||
title: true,
|
||||
excerpt: true,
|
||||
}
|
||||
})
|
||||
|
||||
console.log('Full payload:', JSON.stringify(full).length, 'bytes')
|
||||
console.log('Optimized payload:', JSON.stringify(optimized).length, 'bytes')
|
||||
console.log('Reduction:',
|
||||
Math.round((1 - JSON.stringify(optimized).length / JSON.stringify(full).length) * 100),
|
||||
'%'
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
## Index Verification
|
||||
|
||||
Check that indexes exist for queried fields:
|
||||
|
||||
```sql
|
||||
-- PostgreSQL
|
||||
SELECT indexname, indexdef
|
||||
FROM pg_indexes
|
||||
WHERE tablename = 'Post';
|
||||
|
||||
-- MySQL
|
||||
SHOW INDEXES FROM Post;
|
||||
```
|
||||
|
||||
## Production Monitoring
|
||||
|
||||
Monitor in production:
|
||||
|
||||
1. **APM tools:** Track query performance over time
|
||||
2. **Database metrics:** Monitor slow query log
|
||||
3. **API response times:** Measure endpoint latency
|
||||
4. **Memory usage:** Track application memory consumption
|
||||
|
||||
## Expected Improvements
|
||||
|
||||
After optimization:
|
||||
|
||||
- **Query count:** Reduced to 1-2 queries (from N+1)
|
||||
- **Response size:** 60-90% smaller payload
|
||||
- **Query time:** Similar or faster
|
||||
- **Memory usage:** 50-80% lower
|
||||
101
skills/optimizing-query-selection/references/type-safety.md
Normal file
101
skills/optimizing-query-selection/references/type-safety.md
Normal file
@@ -0,0 +1,101 @@
|
||||
# Type Safety Guide
|
||||
|
||||
## Inferred Types
|
||||
|
||||
TypeScript infers exact return types based on selection:
|
||||
|
||||
```typescript
|
||||
const user = await prisma.user.findUnique({
|
||||
where: { id: 1 },
|
||||
select: {
|
||||
name: true,
|
||||
email: true,
|
||||
posts: {
|
||||
select: {
|
||||
title: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
```
|
||||
|
||||
Inferred type:
|
||||
```typescript
|
||||
{
|
||||
name: string
|
||||
email: string
|
||||
posts: {
|
||||
title: string
|
||||
}[]
|
||||
} | null
|
||||
```
|
||||
|
||||
## Reusable Selection Objects
|
||||
|
||||
Create reusable selection objects:
|
||||
|
||||
```typescript
|
||||
const userBasicSelect = {
|
||||
id: true,
|
||||
name: true,
|
||||
email: true,
|
||||
} as const
|
||||
|
||||
const users = await prisma.user.findMany({
|
||||
select: userBasicSelect,
|
||||
})
|
||||
```
|
||||
|
||||
## Composition Patterns
|
||||
|
||||
Build complex selections from smaller pieces:
|
||||
|
||||
```typescript
|
||||
const authorSelect = {
|
||||
id: true,
|
||||
name: true,
|
||||
email: true,
|
||||
} as const
|
||||
|
||||
const postSelect = {
|
||||
id: true,
|
||||
title: true,
|
||||
author: {
|
||||
select: authorSelect,
|
||||
},
|
||||
} as const
|
||||
|
||||
const posts = await prisma.post.findMany({
|
||||
select: postSelect,
|
||||
})
|
||||
```
|
||||
|
||||
## Type Extraction
|
||||
|
||||
Extract types from selection objects:
|
||||
|
||||
```typescript
|
||||
import { Prisma } from '@prisma/client'
|
||||
|
||||
const postWithAuthor = Prisma.validator<Prisma.PostDefaultArgs>()({
|
||||
select: {
|
||||
id: true,
|
||||
title: true,
|
||||
author: {
|
||||
select: {
|
||||
id: true,
|
||||
name: true,
|
||||
},
|
||||
},
|
||||
},
|
||||
})
|
||||
|
||||
type PostWithAuthor = Prisma.PostGetPayload<typeof postWithAuthor>
|
||||
```
|
||||
|
||||
## Benefits
|
||||
|
||||
- **Type safety:** Compiler catches field typos
|
||||
- **Refactoring:** Changes propagate through types
|
||||
- **Reusability:** Share selection patterns
|
||||
- **Documentation:** Types serve as inline docs
|
||||
Reference in New Issue
Block a user