gh-hopeoverture-worldbuilding-app-skills-plugins-sentry-and-otel-setup/skills/sentry-and-otel-setup/references/otel-integration.md

OpenTelemetry Integration

What is OpenTelemetry?

OpenTelemetry (OTel) is an observability framework for:

• Traces: Track requests through distributed systems
• Metrics: Measure system performance
• Logs: Structured event logging

Sentry acts as an OTel backend, receiving and visualizing this data.

How It Works with Sentry

1. Instrumentation hook (instrumentation.ts) initializes OTel
2. Automatic instrumentation tracks HTTP, database, and framework operations
3. Custom instrumentation adds application-specific tracing
4. Sentry integration sends traces to Sentry for visualization

Automatic Instrumentation

What Gets Traced Automatically

With Sentry + Next.js:

Server-side:

• HTTP requests (incoming/outgoing)
• Database queries (Prisma, PostgreSQL)
• File system operations
• Next.js Server Components
• API routes

Client-side:

• Navigation
• User interactions
• Fetch requests
• Component rendering (partial)

Viewing Traces

In Sentry dashboard:

1. Go to Performance
2. Click on a transaction
3. View waterfall of spans (operations)
4. Identify bottlenecks

Custom Instrumentation

1. Creating Spans

Track specific operations:

import * as Sentry from '@sentry/nextjs';

export async function processWorldData(worldId: string) {
  return await Sentry.startSpan(
    {
      name: 'process-world-data',
      op: 'task',
      attributes: {
        worldId,
      },
    },
    async () => {
      // Your logic here
      const data = await fetchWorldData(worldId);
      const processed = await processData(data);
      return processed;
    }
  );
}

2. Nested Spans

Track sub-operations:

export async function generateWorldContent(worldId: string) {
  return await Sentry.startSpan(
    { name: 'generate-world-content', op: 'task' },
    async (span) => {
      // Child span 1
      const entities = await Sentry.startSpan(
        { name: 'fetch-entities', op: 'db.query' },
        async () => {
          return await prisma.entity.findMany({
            where: { worldId },
          });
        }
      );

      // Child span 2
      const content = await Sentry.startSpan(
        { name: 'generate-content', op: 'ai' },
        async () => {
          return await generateContent(entities);
        }
      );

      span.setStatus('ok');
      return content;
    }
  );
}

3. Adding Attributes

Enrich spans with metadata:

Sentry.startSpan(
  {
    name: 'export-world',
    op: 'export',
    attributes: {
      worldId: 'world-123',
      format: 'json',
      includeAssets: true,
    },
  },
  async (span) => {
    const result = await exportWorld();

    // Add result metadata
    span.setAttributes({
      'export.size': result.size,
      'export.entityCount': result.entityCount,
    });

    return result;
  }
);

Distributed Tracing

Propagating Context

Trace requests across services:

// Service A - initiates request
export async function callServiceB() {
  return await Sentry.startSpan(
    { name: 'call-service-b', op: 'http.client' },
    async () => {
      // Context automatically propagated via headers
      const response = await fetch('https://service-b.com/api', {
        headers: {
          // Sentry automatically adds tracing headers
        },
      });

      return response.json();
    }
  );
}

// Service B - receives request
// Continues the same trace automatically if instrumented

Trace Visualization

In Sentry:

• See complete request flow across services
• Identify slow external calls
• Understand service dependencies

Common Span Operations

Use semantic operation names:

// Database operations
{ op: 'db.query' }
{ op: 'db.insert' }
{ op: 'db.update' }

// HTTP operations
{ op: 'http.client' }
{ op: 'http.server' }

// Business logic
{ op: 'task' }
{ op: 'function' }

// External services
{ op: 'ai' }
{ op: 'cache' }
{ op: 'storage' }

Performance Patterns

1. Identify N+1 Queries

Traces reveal repeated database queries:

// Bad - N+1 problem shows in trace
for (const world of worlds) {
  const entities = await prisma.entity.findMany({
    where: { worldId: world.id },
  });
}
// Trace shows: db.query x N times

// Good - Single query
const entities = await prisma.entity.findMany({
  where: {
    worldId: { in: worlds.map(w => w.id) },
  },
});
// Trace shows: db.query x 1 time

2. Parallel Operations

Visualize parallel vs sequential:

// Sequential - slow
const terrain = await generateTerrain();
const biomes = await generateBiomes();
const climate = await generateClimate();
// Trace: 3 sequential spans

// Parallel - faster
const [terrain, biomes, climate] = await Promise.all([
  generateTerrain(),
  generateBiomes(),
  generateClimate(),
]);
// Trace: 3 parallel spans

3. Caching Effectiveness

Track cache hits/misses:

export async function getCachedData(key: string) {
  return await Sentry.startSpan(
    { name: 'get-cached-data', op: 'cache' },
    async (span) => {
      const cached = await redis.get(key);

      if (cached) {
        span.setAttributes({ 'cache.hit': true });
        return cached;
      }

      span.setAttributes({ 'cache.hit': false });
      const data = await fetchFreshData();
      await redis.set(key, data);
      return data;
    }
  );
}

Sampling Strategies

1. Transaction-Based Sampling

Sample different operations differently:

Sentry.init({
  tracesSampler: (samplingContext) => {
    // High-value operations - trace all
    if (samplingContext.name?.includes('checkout')) {
      return 1.0;
    }

    // Background tasks - sample less
    if (samplingContext.name?.includes('background')) {
      return 0.01;
    }

    // Default
    return 0.1;
  },
});

2. Head-Based Sampling

Decision made at trace start (Sentry default).

Pros: Simple, consistent per trace Cons: Can't sample based on outcome

3. Performance Impact

• Minimal overhead (<5% typically)
• Sampling reduces overhead
• Most cost is in data transmission

Troubleshooting

Traces Not Appearing

1. Check tracesSampleRate > 0
2. Verify instrumentation.ts exports register()
3. Ensure Next.js experimental.instrumentationHook enabled
4. Check Sentry DSN is correct

Missing Spans

1. Verify automatic instrumentation is enabled
2. Check custom spans use correct Sentry API
3. Ensure spans finish (call .finish() or use callback form)

Incomplete Traces

1. Check trace context is propagated (especially cross-service)
2. Verify all services use compatible Sentry SDK versions
3. Review sampling - some spans may be dropped

Best Practices

1. Use semantic operation names - Makes filtering/grouping easier
2. Add relevant attributes - Enrich spans with context
3. Don't over-instrument - Focus on meaningful operations
4. Keep spans short - Track discrete operations, not long processes
5. Set span status - Mark success/failure explicitly
6. Sample appropriately - Balance visibility with cost
7. Review regularly - Check for performance regressions

Advanced: Custom Metrics

Track custom measurements:

import * as Sentry from '@sentry/nextjs';

// Record custom metric
Sentry.metrics.distribution('world.entity.count', entityCount, {
  tags: {
    worldType: 'fantasy',
  },
});

// Track over time in Sentry dashboard

Integration with Other Tools

Sentry can forward traces to:

• DataDog - Full observability platform
• Jaeger - Distributed tracing
• Prometheus - Metrics collection

Configure via Sentry integrations or OpenTelemetry Collector.