gh-tachyon-beep-skillpacks-plugins-axiom-web-backend/skills/using-web-backend/express-development.md

Express Development

Overview

Express.js development specialist covering middleware organization, error handling, validation, database integration, testing, and production deployment.

Core principle: Express's minimalist philosophy requires disciplined patterns - without structure, Express apps become tangled middleware chains with inconsistent error handling and poor testability.

When to Use This Skill

Use when encountering:

• Middleware organization: Ordering, async error handling, custom middleware
• Error handling: Centralized handlers, custom error classes, async/await errors
• Request validation: Zod, express-validator, type-safe validation
• Database patterns: Connection pooling, transactions, graceful shutdown
• Testing: Supertest, mocking, middleware isolation
• Production deployment: PM2, clustering, Docker, environment management
• Performance: Compression, caching, clustering
• Security: Helmet, rate limiting, CORS, input sanitization

DO NOT use for:

• General TypeScript patterns (use axiom-python-engineering equivalents)
• API design principles (use rest-api-design)
• Database-agnostic patterns (use database-integration)

Middleware Organization

Correct Middleware Order

Order matters - middleware executes top to bottom:

import express from 'express';
import helmet from 'helmet';
import cors from 'cors';
import compression from 'compression';

const app = express();

// 1. Security (FIRST - before any parsing)
app.use(helmet({
  contentSecurityPolicy: {
    directives: {
      defaultSrc: ["'self'"],
      styleSrc: ["'self'", "'unsafe-inline'"],
    },
  },
}));

// 2. CORS (before routes)
app.use(cors({
  origin: process.env.ALLOWED_ORIGINS?.split(','),
  credentials: true,
  maxAge: 86400, // 24 hours
}));

// 3. Parsing
app.use(express.json({ limit: '10mb' }));
app.use(express.urlencoded({ extended: true, limit: '10mb' }));

// 4. Compression
app.use(compression());

// 5. Logging
app.use(morgan('combined', { stream: logger.stream }));

// 6. Authentication (before routes that need it)
app.use('/api', authenticationMiddleware);

// 7. Routes
app.use('/api/users', userRoutes);
app.use('/api/posts', postRoutes);

// 8. 404 handler (AFTER all routes)
app.use((req, res) => {
  res.status(404).json({
    status: 'error',
    message: 'Route not found',
    path: req.path,
  });
});

// 9. Error handler (LAST)
app.use(errorHandler);

Async Error Wrapper

Problem: Express doesn't catch async errors automatically

// src/middleware/asyncHandler.ts
import { Request, Response, NextFunction } from 'express';

export const asyncHandler = <T>(
  fn: (req: Request, res: Response, next: NextFunction) => Promise<T>
) => {
  return (req: Request, res: Response, next: NextFunction) => {
    Promise.resolve(fn(req, res, next)).catch(next);
  };
};

// Usage
router.get('/:id', asyncHandler(async (req, res) => {
  const user = await userService.findById(req.params.id);
  if (!user) throw new NotFoundError('User not found');
  res.json(user);
}));

Alternative: Use express-async-errors (automatic)

// At top of app.ts (BEFORE routes)
import 'express-async-errors';

// Now all async route handlers auto-catch errors
router.get('/:id', async (req, res) => {
  const user = await userService.findById(req.params.id);
  res.json(user);
}); // Errors automatically forwarded to error handler

Error Handling

Custom Error Classes

// src/errors/AppError.ts
export class AppError extends Error {
  constructor(
    public readonly message: string,
    public readonly statusCode: number,
    public readonly isOperational: boolean = true
  ) {
    super(message);
    Error.captureStackTrace(this, this.constructor);
  }
}

// src/errors/HttpErrors.ts
export class BadRequestError extends AppError {
  constructor(message: string) {
    super(message, 400);
  }
}

export class UnauthorizedError extends AppError {
  constructor(message = 'Unauthorized') {
    super(message, 401);
  }
}

export class ForbiddenError extends AppError {
  constructor(message = 'Forbidden') {
    super(message, 403);
  }
}

export class NotFoundError extends AppError {
  constructor(message: string) {
    super(message, 404);
  }
}

export class ConflictError extends AppError {
  constructor(message: string) {
    super(message, 409);
  }
}

export class TooManyRequestsError extends AppError {
  constructor(message = 'Too many requests', public retryAfter?: number) {
    super(message, 429);
  }
}

Centralized Error Handler

// src/middleware/errorHandler.ts
import { Request, Response, NextFunction } from 'express';
import { AppError } from '../errors/AppError';
import { logger } from '../config/logger';

export const errorHandler = (
  err: Error,
  req: Request,
  res: Response,
  next: NextFunction
) => {
  // Log error with context
  logger.error('Error occurred', {
    error: {
      message: err.message,
      stack: err.stack,
      name: err.name,
    },
    request: {
      method: req.method,
      url: req.url,
      ip: req.ip,
      userAgent: req.get('user-agent'),
    },
  });

  // Operational errors (expected)
  if (err instanceof AppError && err.isOperational) {
    const response: any = {
      status: 'error',
      message: err.message,
    };

    // Add retry-after for rate limiting
    if (err instanceof TooManyRequestsError && err.retryAfter) {
      res.setHeader('Retry-After', err.retryAfter);
      response.retryAfter = err.retryAfter;
    }

    return res.status(err.statusCode).json(response);
  }

  // Validation errors (Zod, express-validator)
  if (err.name === 'ZodError') {
    return res.status(400).json({
      status: 'error',
      message: 'Validation failed',
      errors: (err as any).errors,
    });
  }

  // Database constraint violations
  if ((err as any).code === '23505') { // PostgreSQL unique violation
    return res.status(409).json({
      status: 'error',
      message: 'Resource already exists',
    });
  }

  if ((err as any).code === '23503') { // Foreign key violation
    return res.status(400).json({
      status: 'error',
      message: 'Invalid reference',
    });
  }

  // Unexpected errors (don't leak details in production)
  res.status(500).json({
    status: 'error',
    message: process.env.NODE_ENV === 'production'
      ? 'Internal server error'
      : err.message,
    ...(process.env.NODE_ENV !== 'production' && { stack: err.stack }),
  });
};

Global Error Handlers

// src/server.ts
process.on('unhandledRejection', (reason: Error) => {
  logger.error('Unhandled Rejection', { reason });
  // Graceful shutdown
  server.close(() => process.exit(1));
});

process.on('uncaughtException', (error: Error) => {
  logger.error('Uncaught Exception', { error });
  process.exit(1);
});

Request Validation

Zod Integration (Type-Safe)

// src/schemas/userSchema.ts
import { z } from 'zod';

export const createUserSchema = z.object({
  body: z.object({
    email: z.string().email('Invalid email'),
    password: z.string()
      .min(8, 'Password must be at least 8 characters')
      .regex(/[A-Z]/, 'Password must contain uppercase')
      .regex(/[0-9]/, 'Password must contain number'),
    name: z.string().min(2).max(100),
    age: z.number().int().positive().max(150).optional(),
  }),
});

export const getUserSchema = z.object({
  params: z.object({
    id: z.string().regex(/^\d+$/, 'ID must be numeric'),
  }),
});

export const getUsersSchema = z.object({
  query: z.object({
    page: z.string().regex(/^\d+$/).transform(Number).default('1'),
    limit: z.string().regex(/^\d+$/).transform(Number).default('10'),
    search: z.string().optional(),
    sortBy: z.enum(['name', 'created_at', 'updated_at']).optional(),
    order: z.enum(['asc', 'desc']).optional(),
  }),
});

// Type inference
export type CreateUserInput = z.infer<typeof createUserSchema>['body'];
export type GetUserParams = z.infer<typeof getUserSchema>['params'];
export type GetUsersQuery = z.infer<typeof getUsersSchema>['query'];

Validation middleware:

// src/middleware/validate.ts
import { Request, Response, NextFunction } from 'express';
import { AnyZodObject, ZodError } from 'zod';

export const validate = (schema: AnyZodObject) => {
  return async (req: Request, res: Response, next: NextFunction) => {
    try {
      const validated = await schema.parseAsync({
        body: req.body,
        query: req.query,
        params: req.params,
      });

      // Replace with validated data (transforms applied)
      req.body = validated.body || req.body;
      req.query = validated.query || req.query;
      req.params = validated.params || req.params;

      next();
    } catch (error) {
      if (error instanceof ZodError) {
        return res.status(400).json({
          status: 'error',
          message: 'Validation failed',
          errors: error.errors.map(err => ({
            field: err.path.join('.'),
            message: err.message,
            code: err.code,
          })),
        });
      }
      next(error);
    }
  };
};

Usage in routes:

import { Router } from 'express';
import { validate } from '../middleware/validate';
import * as schemas from '../schemas/userSchema';

const router = Router();

router.post('/', validate(schemas.createUserSchema), async (req, res) => {
  // req.body is now typed as CreateUserInput
  const user = await userService.create(req.body);
  res.status(201).json(user);
});

router.get('/:id', validate(schemas.getUserSchema), async (req, res) => {
  // req.params.id is validated
  const user = await userService.findById(req.params.id);
  if (!user) throw new NotFoundError('User not found');
  res.json(user);
});

Database Connection Pooling

PostgreSQL with pg

// src/config/database.ts
import { Pool, PoolConfig } from 'pg';
import { logger } from './logger';

const config: PoolConfig = {
  host: process.env.DB_HOST || 'localhost',
  port: Number(process.env.DB_PORT) || 5432,
  database: process.env.DB_NAME,
  user: process.env.DB_USER,
  password: process.env.DB_PASSWORD,
  max: Number(process.env.DB_POOL_MAX) || 20,
  idleTimeoutMillis: 30000,
  connectionTimeoutMillis: 2000,
  statement_timeout: 30000, // 30s query timeout
};

export const pool = new Pool(config);

// Event handlers
pool.on('connect', (client) => {
  logger.debug('Database client connected');
});

pool.on('acquire', (client) => {
  logger.debug('Client acquired from pool');
});

pool.on('error', (err, client) => {
  logger.error('Unexpected pool error', { error: err });
  process.exit(-1);
});

// Health check
export const testConnection = async () => {
  try {
    const client = await pool.connect();
    const result = await client.query('SELECT NOW()');
    client.release();
    logger.info('Database connection successful', {
      serverTime: result.rows[0].now,
    });
  } catch (err) {
    logger.error('Database connection failed', { error: err });
    throw err;
  }
};

// Graceful shutdown
export const closePool = async () => {
  logger.info('Closing database pool');
  await pool.end();
  logger.info('Database pool closed');
};

Transaction Helper

// src/utils/transaction.ts
import { Pool, PoolClient } from 'pg';

export async function withTransaction<T>(
  pool: Pool,
  callback: (client: PoolClient) => Promise<T>
): Promise<T> {
  const client = await pool.connect();

  try {
    await client.query('BEGIN');
    const result = await callback(client);
    await client.query('COMMIT');
    return result;
  } catch (error) {
    await client.query('ROLLBACK');
    throw error;
  } finally {
    client.release();
  }
}

// Usage
import { pool } from '../config/database';

async function createUserWithProfile(userData, profileData) {
  return withTransaction(pool, async (client) => {
    const userResult = await client.query(
      'INSERT INTO users (email, name) VALUES ($1, $2) RETURNING id',
      [userData.email, userData.name]
    );
    const userId = userResult.rows[0].id;

    await client.query(
      'INSERT INTO profiles (user_id, bio) VALUES ($1, $2)',
      [userId, profileData.bio]
    );

    return userId;
  });
}

Testing

Integration Tests with Supertest

// tests/integration/userRoutes.test.ts
import request from 'supertest';
import app from '../../src/app';
import { pool } from '../../src/config/database';

describe('User Routes', () => {
  beforeAll(async () => {
    await pool.query('CREATE TABLE IF NOT EXISTS users (...)');
  });

  afterEach(async () => {
    await pool.query('TRUNCATE TABLE users CASCADE');
  });

  afterAll(async () => {
    await pool.end();
  });

  describe('POST /api/users', () => {
    it('should create user with valid data', async () => {
      const response = await request(app)
        .post('/api/users')
        .send({
          email: 'test@example.com',
          name: 'Test User',
          password: 'Password123',
        })
        .expect(201);

      expect(response.body).toHaveProperty('id');
      expect(response.body.email).toBe('test@example.com');
      expect(response.body).not.toHaveProperty('password');
    });

    it('should return 400 for invalid email', async () => {
      const response = await request(app)
        .post('/api/users')
        .send({
          email: 'invalid',
          name: 'Test',
          password: 'Password123',
        })
        .expect(400);

      expect(response.body.status).toBe('error');
      expect(response.body.errors).toContainEqual(
        expect.objectContaining({
          field: 'body.email',
          message: expect.stringContaining('email'),
        })
      );
    });
  });

  describe('GET /api/users/:id', () => {
    it('should return user by ID', async () => {
      const createRes = await request(app)
        .post('/api/users')
        .send({
          email: 'test@example.com',
          name: 'Test User',
          password: 'Password123',
        });

      const response = await request(app)
        .get(`/api/users/${createRes.body.id}`)
        .expect(200);

      expect(response.body.id).toBe(createRes.body.id);
    });

    it('should return 404 for non-existent user', async () => {
      await request(app)
        .get('/api/users/99999')
        .expect(404);
    });
  });
});

Unit Tests with Mocks

// tests/unit/userService.test.ts
import { userService } from '../../src/services/userService';
import { pool } from '../../src/config/database';

jest.mock('../../src/config/database');

const mockPool = pool as jest.Mocked<typeof pool>;

describe('UserService', () => {
  beforeEach(() => {
    jest.clearAllMocks();
  });

  describe('findById', () => {
    it('should return user when found', async () => {
      mockPool.query.mockResolvedValue({
        rows: [{ id: 1, email: 'test@example.com', name: 'Test' }],
        command: 'SELECT',
        rowCount: 1,
        oid: 0,
        fields: [],
      });

      const result = await userService.findById('1');

      expect(result).toEqual(
        expect.objectContaining({ id: 1, email: 'test@example.com' })
      );
    });

    it('should return null when not found', async () => {
      mockPool.query.mockResolvedValue({
        rows: [],
        command: 'SELECT',
        rowCount: 0,
        oid: 0,
        fields: [],
      });

      const result = await userService.findById('999');
      expect(result).toBeNull();
    });
  });
});

Production Deployment

PM2 Configuration

// ecosystem.config.js
module.exports = {
  apps: [{
    name: 'api',
    script: './dist/server.js',
    instances: 'max', // Use all CPU cores
    exec_mode: 'cluster',
    env: {
      NODE_ENV: 'production',
      PORT: 3000,
    },
    error_file: './logs/err.log',
    out_file: './logs/out.log',
    log_date_format: 'YYYY-MM-DD HH:mm:ss Z',
    merge_logs: true,
    max_memory_restart: '500M',
    wait_ready: true,
    listen_timeout: 10000,
    kill_timeout: 5000,
  }],
};

Graceful shutdown with PM2:

// src/server.ts
const server = app.listen(PORT, () => {
  logger.info(`Server started on port ${PORT}`);

  // Signal PM2 ready
  if (process.send) {
    process.send('ready');
  }
});

// Graceful shutdown
process.on('SIGINT', async () => {
  logger.info('SIGINT received, closing server');

  server.close(async () => {
    await closePool();
    logger.info('Server closed');
    process.exit(0);
  });

  // Force shutdown after 10s
  setTimeout(() => {
    logger.error('Forcing shutdown');
    process.exit(1);
  }, 10000);
});

Dockerfile

# Multi-stage build
FROM node:18-alpine AS builder

WORKDIR /app

# Copy package files
COPY package*.json ./
COPY tsconfig.json ./

# Install dependencies
RUN npm ci

# Copy source
COPY src ./src

# Build TypeScript
RUN npm run build

# Production image
FROM node:18-alpine

WORKDIR /app

# Install production dependencies only
COPY package*.json ./
RUN npm ci --omit=dev && npm cache clean --force

# Copy built files
COPY --from=builder /app/dist ./dist

# Create non-root user
RUN addgroup -g 1001 -S nodejs && \
    adduser -S nodejs -u 1001

USER nodejs

EXPOSE 3000

CMD ["node", "dist/server.js"]

Health Check Endpoint

// src/routes/healthRoutes.ts
import { Router } from 'express';
import { pool } from '../config/database';

const router = Router();

router.get('/health', async (req, res) => {
  const health = {
    uptime: process.uptime(),
    message: 'OK',
    timestamp: Date.now(),
  };

  try {
    await pool.query('SELECT 1');
    health.database = 'connected';
  } catch (error) {
    health.database = 'disconnected';
    return res.status(503).json(health);
  }

  res.json(health);
});

router.get('/health/ready', async (req, res) => {
  // Readiness check
  try {
    await pool.query('SELECT 1');
    res.status(200).json({ status: 'ready' });
  } catch (error) {
    res.status(503).json({ status: 'not ready' });
  }
});

router.get('/health/live', (req, res) => {
  // Liveness check (simpler)
  res.status(200).json({ status: 'alive' });
});

export default router;

Performance Optimization

Response Caching

import Redis from 'ioredis';

const redis = new Redis({
  host: process.env.REDIS_HOST,
  port: Number(process.env.REDIS_PORT),
});

export const cacheMiddleware = (duration: number) => {
  return async (req: Request, res: Response, next: NextFunction) => {
    if (req.method !== 'GET') return next();

    const key = `cache:${req.originalUrl}`;

    try {
      const cached = await redis.get(key);
      if (cached) {
        return res.json(JSON.parse(cached));
      }

      // Capture response
      const originalJson = res.json.bind(res);
      res.json = (body: any) => {
        redis.setex(key, duration, JSON.stringify(body));
        return originalJson(body);
      };

      next();
    } catch (error) {
      next();
    }
  };
};

// Usage
router.get('/users', cacheMiddleware(300), async (req, res) => {
  const users = await userService.findAll();
  res.json(users);
});

Security

Rate Limiting

import rateLimit from 'express-rate-limit';
import RedisStore from 'rate-limit-redis';
import Redis from 'ioredis';

const redis = new Redis();

export const apiLimiter = rateLimit({
  store: new RedisStore({ client: redis }),
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 100, // 100 requests per window
  message: 'Too many requests, please try again later',
  standardHeaders: true,
  legacyHeaders: false,
});

export const authLimiter = rateLimit({
  store: new RedisStore({ client: redis }),
  windowMs: 15 * 60 * 1000,
  max: 5, // 5 attempts
  skipSuccessfulRequests: true,
});

// Usage
app.use('/api/', apiLimiter);
app.use('/api/auth/login', authLimiter);

Anti-Patterns

Anti-Pattern	Why Bad	Fix
No async error handling	Crashes server	Use asyncHandler or express-async-errors
Inconsistent error responses	Poor DX	Centralized error handler
New DB connection per request	Exhausts connections	Use connection pool
No graceful shutdown	Data loss, broken requests	Handle SIGTERM/SIGINT
Logging to console in production	Lost logs, no structure	Use Winston/Pino with transports
No request validation	Security vulnerabilities	Zod/express-validator
Synchronous operations in routes	Blocks event loop	Use async/await
No health checks	Can't monitor service	/health endpoints

Cross-References

Related skills:

• Database patterns → database-integration (pooling, transactions)
• API testing → api-testing (supertest patterns)
• REST design → rest-api-design (endpoint patterns)
• Authentication → api-authentication (JWT, sessions)

Further Reading

• Express docs: https://expressjs.com/
• Express.js Best Practices: https://expressjs.com/en/advanced/best-practice-performance.html
• Node.js Production Best Practices: https://github.com/goldbergyoni/nodebestpractices