Initial commit
This commit is contained in:
632
references/common-errors.md
Normal file
632
references/common-errors.md
Normal file
@@ -0,0 +1,632 @@
|
||||
# Common Errors and Solutions
|
||||
|
||||
Complete reference for all known Browser Rendering errors with sources, root causes, and solutions.
|
||||
|
||||
---
|
||||
|
||||
## Error 1: "Cannot read properties of undefined (reading 'fetch')"
|
||||
|
||||
**Full Error:**
|
||||
```
|
||||
TypeError: Cannot read properties of undefined (reading 'fetch')
|
||||
```
|
||||
|
||||
**Source**: https://developers.cloudflare.com/browser-rendering/faq/#cannot-read-properties-of-undefined-reading-fetch
|
||||
|
||||
**Root Cause**: Browser binding not passed to `puppeteer.launch()`
|
||||
|
||||
**Why It Happens:**
|
||||
```typescript
|
||||
// ❌ Missing browser binding
|
||||
const browser = await puppeteer.launch();
|
||||
// ^ undefined - no binding passed!
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```typescript
|
||||
// ✅ Pass browser binding
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
// ^^^^^^^^^^^^^^^^ binding from env
|
||||
```
|
||||
|
||||
**Prevention**: Always pass `env.MYBROWSER` (or your configured binding name) to `puppeteer.launch()`.
|
||||
|
||||
---
|
||||
|
||||
## Error 2: XPath Selector Not Supported
|
||||
|
||||
**Full Error:**
|
||||
```
|
||||
Error: XPath selectors are not supported in Browser Rendering
|
||||
```
|
||||
|
||||
**Source**: https://developers.cloudflare.com/browser-rendering/faq/#why-cant-i-use-an-xpath-selector-when-using-browser-rendering-with-puppeteer
|
||||
|
||||
**Root Cause**: XPath poses security risk to Workers
|
||||
|
||||
**Why It Happens:**
|
||||
```typescript
|
||||
// ❌ XPath selectors not directly supported
|
||||
const elements = await page.$x('/html/body/div/h1');
|
||||
```
|
||||
|
||||
**Solution 1: Use CSS Selectors**
|
||||
```typescript
|
||||
// ✅ Use CSS selector instead
|
||||
const element = await page.$("div > h1");
|
||||
const elements = await page.$$("div > h1");
|
||||
```
|
||||
|
||||
**Solution 2: Use XPath in page.evaluate()**
|
||||
```typescript
|
||||
// ✅ Use XPath inside page.evaluate()
|
||||
const innerHtml = await page.evaluate(() => {
|
||||
return (
|
||||
// @ts-ignore - runs in browser context
|
||||
new XPathEvaluator()
|
||||
.createExpression("/html/body/div/h1")
|
||||
// @ts-ignore
|
||||
.evaluate(document, XPathResult.FIRST_ORDERED_NODE_TYPE)
|
||||
.singleNodeValue.innerHTML
|
||||
);
|
||||
});
|
||||
```
|
||||
|
||||
**Prevention**: Use CSS selectors by default. Only use XPath via `page.evaluate()` if absolutely necessary.
|
||||
|
||||
---
|
||||
|
||||
## Error 3: Browser Timeout
|
||||
|
||||
**Full Error:**
|
||||
```
|
||||
Error: Browser session closed due to inactivity
|
||||
```
|
||||
|
||||
**Source**: https://developers.cloudflare.com/browser-rendering/platform/limits/#note-on-browser-timeout
|
||||
|
||||
**Root Cause**: Default 60 second idle timeout
|
||||
|
||||
**Why It Happens:**
|
||||
- No devtools commands sent for 60 seconds
|
||||
- Browser automatically closes to free resources
|
||||
|
||||
**Solution: Extend Timeout**
|
||||
```typescript
|
||||
// ✅ Extend timeout to 5 minutes
|
||||
const browser = await puppeteer.launch(env.MYBROWSER, {
|
||||
keep_alive: 300000 // 5 minutes = 300,000 ms
|
||||
});
|
||||
```
|
||||
|
||||
**Maximum**: 600,000ms (10 minutes)
|
||||
|
||||
**Use Cases for Extended Timeout:**
|
||||
- Multi-step workflows
|
||||
- Long-running scraping
|
||||
- Session reuse across requests
|
||||
|
||||
**Prevention**: Only extend if actually needed. Longer timeout = more billable hours.
|
||||
|
||||
---
|
||||
|
||||
## Error 4: Rate Limit Exceeded
|
||||
|
||||
**Full Error:**
|
||||
```
|
||||
Error: Rate limit exceeded. Too many concurrent browsers.
|
||||
```
|
||||
|
||||
**Source**: https://developers.cloudflare.com/browser-rendering/platform/limits/
|
||||
|
||||
**Root Cause**: Exceeded concurrent browser limit
|
||||
|
||||
**Limits:**
|
||||
- Free tier: 3 concurrent browsers
|
||||
- Paid tier: 10-30 concurrent browsers
|
||||
|
||||
**Solution 1: Check Limits Before Launching**
|
||||
```typescript
|
||||
const limits = await puppeteer.limits(env.MYBROWSER);
|
||||
|
||||
if (limits.allowedBrowserAcquisitions === 0) {
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
error: "Rate limit reached",
|
||||
retryAfter: limits.timeUntilNextAllowedBrowserAcquisition
|
||||
}),
|
||||
{ status: 429 }
|
||||
);
|
||||
}
|
||||
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
```
|
||||
|
||||
**Solution 2: Reuse Sessions**
|
||||
```typescript
|
||||
// Try to connect to existing session first
|
||||
const sessions = await puppeteer.sessions(env.MYBROWSER);
|
||||
const freeSession = sessions.find(s => !s.connectionId);
|
||||
|
||||
if (freeSession) {
|
||||
try {
|
||||
return await puppeteer.connect(env.MYBROWSER, freeSession.sessionId);
|
||||
} catch {
|
||||
// Session closed, launch new
|
||||
}
|
||||
}
|
||||
|
||||
return await puppeteer.launch(env.MYBROWSER);
|
||||
```
|
||||
|
||||
**Solution 3: Use Multiple Tabs**
|
||||
```typescript
|
||||
// ❌ Bad: 10 browsers
|
||||
for (const url of urls) {
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
// ...
|
||||
}
|
||||
|
||||
// ✅ Good: 1 browser, 10 tabs
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
await Promise.all(urls.map(async url => {
|
||||
const page = await browser.newPage();
|
||||
// ...
|
||||
await page.close();
|
||||
}));
|
||||
await browser.close();
|
||||
```
|
||||
|
||||
**Prevention**: Monitor concurrency usage, implement session reuse, use tabs instead of multiple browsers.
|
||||
|
||||
---
|
||||
|
||||
## Error 5: Local Development Request Size Limit
|
||||
|
||||
**Full Error:**
|
||||
```
|
||||
Error: Request payload too large (>1MB)
|
||||
```
|
||||
|
||||
**Source**: https://developers.cloudflare.com/browser-rendering/faq/#does-local-development-support-all-browser-rendering-features
|
||||
|
||||
**Root Cause**: Local development limitation (requests >1MB fail)
|
||||
|
||||
**Solution: Use Remote Binding**
|
||||
```jsonc
|
||||
// wrangler.jsonc
|
||||
{
|
||||
"browser": {
|
||||
"binding": "MYBROWSER",
|
||||
"remote": true // ← Use real headless browser during dev
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**With Remote Binding:**
|
||||
- Connects to actual Cloudflare browser (not local simulation)
|
||||
- No 1MB request limit
|
||||
- Counts toward your quota
|
||||
|
||||
**Prevention**: Enable `remote: true` for local development if working with large payloads.
|
||||
|
||||
---
|
||||
|
||||
## Error 6: Bot Protection Triggered
|
||||
|
||||
**Full Error:**
|
||||
```
|
||||
Blocked by bot protection / CAPTCHA challenge
|
||||
```
|
||||
|
||||
**Source**: https://developers.cloudflare.com/browser-rendering/faq/#will-browser-rendering-bypass-cloudflares-bot-protection
|
||||
|
||||
**Root Cause**: Browser Rendering requests always identified as bots
|
||||
|
||||
**Why It Happens:**
|
||||
- Cloudflare automatically identifies Browser Rendering traffic
|
||||
- Cannot bypass bot protection
|
||||
- Automatic headers added: `cf-biso-request-id`, `cf-biso-devtools`
|
||||
|
||||
**Solution (If Scraping Your Own Zone):**
|
||||
Create WAF skip rule:
|
||||
|
||||
1. Go to Security > WAF > Custom rules
|
||||
2. Create skip rule with custom header:
|
||||
- Header: `X-Custom-Auth`
|
||||
- Value: `your-secret-token`
|
||||
3. Add header in your Worker:
|
||||
|
||||
```typescript
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
const page = await browser.newPage();
|
||||
|
||||
// Set custom header
|
||||
await page.setExtraHTTPHeaders({
|
||||
"X-Custom-Auth": "your-secret-token"
|
||||
});
|
||||
|
||||
await page.goto(url);
|
||||
```
|
||||
|
||||
**Solution (If Scraping External Sites):**
|
||||
- Cannot bypass bot protection
|
||||
- Some sites will block Browser Rendering traffic
|
||||
- Consider using site's official API instead
|
||||
|
||||
**Prevention**: Use official APIs when available. Only scrape your own zones if possible.
|
||||
|
||||
---
|
||||
|
||||
## Error 7: Navigation Timeout
|
||||
|
||||
**Full Error:**
|
||||
```
|
||||
TimeoutError: Navigation timeout of 30000 ms exceeded
|
||||
```
|
||||
|
||||
**Root Cause**: Page failed to load within timeout
|
||||
|
||||
**Why It Happens:**
|
||||
- Slow website
|
||||
- Large page assets
|
||||
- Network issues
|
||||
- Page never reaches desired load state
|
||||
|
||||
**Solution 1: Increase Timeout**
|
||||
```typescript
|
||||
await page.goto(url, {
|
||||
timeout: 60000 // 60 seconds
|
||||
});
|
||||
```
|
||||
|
||||
**Solution 2: Change Wait Condition**
|
||||
```typescript
|
||||
// ❌ Strict (waits for all network requests)
|
||||
await page.goto(url, { waitUntil: "networkidle0" });
|
||||
|
||||
// ✅ More lenient (waits for DOMContentLoaded)
|
||||
await page.goto(url, { waitUntil: "domcontentloaded" });
|
||||
|
||||
// ✅ Most lenient (waits for load event only)
|
||||
await page.goto(url, { waitUntil: "load" });
|
||||
```
|
||||
|
||||
**Solution 3: Handle Timeout Gracefully**
|
||||
```typescript
|
||||
try {
|
||||
await page.goto(url, { timeout: 30000 });
|
||||
} catch (error) {
|
||||
if (error instanceof Error && error.name === "TimeoutError") {
|
||||
console.log("Navigation timeout, taking screenshot anyway");
|
||||
const screenshot = await page.screenshot();
|
||||
return screenshot;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
```
|
||||
|
||||
**Prevention**: Set appropriate timeouts for your use case. Use lenient wait conditions for slow sites.
|
||||
|
||||
---
|
||||
|
||||
## Error 8: Memory Limit Exceeded
|
||||
|
||||
**Full Error:**
|
||||
```
|
||||
Error: Browser exceeded its memory limit
|
||||
```
|
||||
|
||||
**Root Cause**: Page too large or too many tabs open
|
||||
|
||||
**Why It Happens:**
|
||||
- Opening many tabs simultaneously
|
||||
- Large pages with many assets
|
||||
- Memory leaks from not closing pages
|
||||
|
||||
**Solution 1: Close Pages**
|
||||
```typescript
|
||||
const page = await browser.newPage();
|
||||
// ... use page ...
|
||||
await page.close(); // ← Don't forget!
|
||||
```
|
||||
|
||||
**Solution 2: Limit Concurrent Tabs**
|
||||
```typescript
|
||||
import PQueue from "p-queue";
|
||||
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
const queue = new PQueue({ concurrency: 5 }); // Max 5 tabs
|
||||
|
||||
await Promise.all(urls.map(url =>
|
||||
queue.add(async () => {
|
||||
const page = await browser.newPage();
|
||||
await page.goto(url);
|
||||
// ...
|
||||
await page.close();
|
||||
})
|
||||
));
|
||||
```
|
||||
|
||||
**Solution 3: Use Smaller Viewports**
|
||||
```typescript
|
||||
await page.setViewport({
|
||||
width: 1280,
|
||||
height: 720 // Smaller than default
|
||||
});
|
||||
```
|
||||
|
||||
**Prevention**: Always close pages when done. Limit concurrent tabs. Process URLs in batches.
|
||||
|
||||
---
|
||||
|
||||
## Error 9: Failed to Connect to Session
|
||||
|
||||
**Full Error:**
|
||||
```
|
||||
Error: Failed to connect to browser session
|
||||
```
|
||||
|
||||
**Root Cause**: Session closed between `.sessions()` and `.connect()` calls
|
||||
|
||||
**Why It Happens:**
|
||||
- Session timed out (60s idle)
|
||||
- Session closed by another Worker
|
||||
- Session terminated unexpectedly
|
||||
|
||||
**Solution: Handle Connection Failures**
|
||||
```typescript
|
||||
const sessions = await puppeteer.sessions(env.MYBROWSER);
|
||||
const freeSession = sessions.find(s => !s.connectionId);
|
||||
|
||||
if (freeSession) {
|
||||
try {
|
||||
const browser = await puppeteer.connect(env.MYBROWSER, freeSession.sessionId);
|
||||
return browser;
|
||||
} catch (error) {
|
||||
console.log("Failed to connect to session, launching new browser");
|
||||
}
|
||||
}
|
||||
|
||||
// Fall back to launching new browser
|
||||
return await puppeteer.launch(env.MYBROWSER);
|
||||
```
|
||||
|
||||
**Prevention**: Always wrap `puppeteer.connect()` in try-catch. Have fallback to `puppeteer.launch()`.
|
||||
|
||||
---
|
||||
|
||||
## Error 10: Too Many Requests Per Minute
|
||||
|
||||
**Full Error:**
|
||||
```
|
||||
Error: Too many browser launches per minute
|
||||
```
|
||||
|
||||
**Root Cause**: Exceeded "new browsers per minute" limit
|
||||
|
||||
**Limits:**
|
||||
- Free tier: 3 per minute (1 every 20 seconds)
|
||||
- Paid tier: 30 per minute (1 every 2 seconds)
|
||||
|
||||
**Solution: Implement Rate Limiting**
|
||||
```typescript
|
||||
async function launchWithRateLimit(env: Env): Promise<Browser> {
|
||||
const limits = await puppeteer.limits(env.MYBROWSER);
|
||||
|
||||
if (limits.allowedBrowserAcquisitions === 0) {
|
||||
const delay = limits.timeUntilNextAllowedBrowserAcquisition || 2000;
|
||||
console.log(`Rate limited, waiting ${delay}ms`);
|
||||
await new Promise(resolve => setTimeout(resolve, delay));
|
||||
}
|
||||
|
||||
return await puppeteer.launch(env.MYBROWSER);
|
||||
}
|
||||
```
|
||||
|
||||
**Prevention**: Check limits before launching. Implement exponential backoff. Reuse sessions instead of launching new browsers.
|
||||
|
||||
---
|
||||
|
||||
## Error 11: Binding Not Configured
|
||||
|
||||
**Full Error:**
|
||||
```
|
||||
Error: Browser binding not found
|
||||
```
|
||||
|
||||
**Root Cause**: Browser binding not configured in wrangler.jsonc
|
||||
|
||||
**Solution: Add Browser Binding**
|
||||
```jsonc
|
||||
// wrangler.jsonc
|
||||
{
|
||||
"browser": {
|
||||
"binding": "MYBROWSER"
|
||||
},
|
||||
"compatibility_flags": ["nodejs_compat"]
|
||||
}
|
||||
```
|
||||
|
||||
**Also Add to TypeScript Types:**
|
||||
```typescript
|
||||
interface Env {
|
||||
MYBROWSER: Fetcher;
|
||||
}
|
||||
```
|
||||
|
||||
**Prevention**: Always configure browser binding and nodejs_compat flag.
|
||||
|
||||
---
|
||||
|
||||
## Error 12: nodejs_compat Flag Missing
|
||||
|
||||
**Full Error:**
|
||||
```
|
||||
Error: Node.js APIs not available
|
||||
```
|
||||
|
||||
**Root Cause**: `nodejs_compat` compatibility flag not enabled
|
||||
|
||||
**Solution: Add Compatibility Flag**
|
||||
```jsonc
|
||||
// wrangler.jsonc
|
||||
{
|
||||
"compatibility_flags": ["nodejs_compat"]
|
||||
}
|
||||
```
|
||||
|
||||
**Why It's Required:**
|
||||
Browser Rendering needs Node.js APIs and polyfills to work.
|
||||
|
||||
**Prevention**: Always include `nodejs_compat` when using Browser Rendering.
|
||||
|
||||
---
|
||||
|
||||
## Error Handling Template
|
||||
|
||||
Complete error handling for production use:
|
||||
|
||||
```typescript
|
||||
import puppeteer, { Browser } from "@cloudflare/puppeteer";
|
||||
|
||||
interface Env {
|
||||
MYBROWSER: Fetcher;
|
||||
}
|
||||
|
||||
async function withBrowser<T>(
|
||||
env: Env,
|
||||
fn: (browser: Browser) => Promise<T>
|
||||
): Promise<T> {
|
||||
let browser: Browser | null = null;
|
||||
|
||||
try {
|
||||
// Check limits
|
||||
const limits = await puppeteer.limits(env.MYBROWSER);
|
||||
if (limits.allowedBrowserAcquisitions === 0) {
|
||||
throw new Error(
|
||||
`Rate limit reached. Retry after ${limits.timeUntilNextAllowedBrowserAcquisition}ms`
|
||||
);
|
||||
}
|
||||
|
||||
// Try to reuse session
|
||||
const sessions = await puppeteer.sessions(env.MYBROWSER);
|
||||
const freeSession = sessions.find(s => !s.connectionId);
|
||||
|
||||
if (freeSession) {
|
||||
try {
|
||||
browser = await puppeteer.connect(env.MYBROWSER, freeSession.sessionId);
|
||||
} catch (error) {
|
||||
console.log("Failed to connect, launching new browser");
|
||||
browser = await puppeteer.launch(env.MYBROWSER);
|
||||
}
|
||||
} else {
|
||||
browser = await puppeteer.launch(env.MYBROWSER);
|
||||
}
|
||||
|
||||
// Execute user function
|
||||
const result = await fn(browser);
|
||||
|
||||
// Disconnect (keep session alive)
|
||||
await browser.disconnect();
|
||||
|
||||
return result;
|
||||
} catch (error) {
|
||||
// Close on error
|
||||
if (browser) {
|
||||
await browser.close();
|
||||
}
|
||||
|
||||
// Re-throw with context
|
||||
if (error instanceof Error) {
|
||||
error.message = `Browser operation failed: ${error.message}`;
|
||||
}
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
export default {
|
||||
async fetch(request: Request, env: Env): Promise<Response> {
|
||||
try {
|
||||
const screenshot = await withBrowser(env, async (browser) => {
|
||||
const page = await browser.newPage();
|
||||
|
||||
try {
|
||||
await page.goto("https://example.com", {
|
||||
waitUntil: "networkidle0",
|
||||
timeout: 30000
|
||||
});
|
||||
} catch (error) {
|
||||
if (error instanceof Error && error.name === "TimeoutError") {
|
||||
console.log("Navigation timeout, taking screenshot anyway");
|
||||
} else {
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
|
||||
return await page.screenshot();
|
||||
});
|
||||
|
||||
return new Response(screenshot, {
|
||||
headers: { "content-type": "image/png" }
|
||||
});
|
||||
} catch (error) {
|
||||
console.error("Request failed:", error);
|
||||
|
||||
return new Response(
|
||||
JSON.stringify({
|
||||
error: error instanceof Error ? error.message : "Unknown error"
|
||||
}),
|
||||
{
|
||||
status: 500,
|
||||
headers: { "content-type": "application/json" }
|
||||
}
|
||||
);
|
||||
}
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Debugging Checklist
|
||||
|
||||
When encountering browser errors:
|
||||
|
||||
1. **Check browser binding**
|
||||
- [ ] Binding configured in wrangler.jsonc?
|
||||
- [ ] nodejs_compat flag enabled?
|
||||
- [ ] Binding passed to puppeteer.launch()?
|
||||
|
||||
2. **Check limits**
|
||||
- [ ] Within concurrent browser limit?
|
||||
- [ ] Within new browsers/minute limit?
|
||||
- [ ] Call puppeteer.limits() to verify?
|
||||
|
||||
3. **Check timeouts**
|
||||
- [ ] Navigation timeout appropriate?
|
||||
- [ ] Browser keep_alive set if needed?
|
||||
- [ ] Timeout errors handled gracefully?
|
||||
|
||||
4. **Check session management**
|
||||
- [ ] browser.close() called on errors?
|
||||
- [ ] Pages closed when done?
|
||||
- [ ] Session reuse implemented correctly?
|
||||
|
||||
5. **Check network**
|
||||
- [ ] Target URL accessible?
|
||||
- [ ] No CORS/bot protection issues?
|
||||
- [ ] Appropriate wait conditions used?
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- **FAQ**: https://developers.cloudflare.com/browser-rendering/faq/
|
||||
- **Limits**: https://developers.cloudflare.com/browser-rendering/platform/limits/
|
||||
- **GitHub Issues**: https://github.com/cloudflare/puppeteer/issues
|
||||
- **Discord**: https://discord.cloudflare.com/
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-10-22
|
||||
593
references/pricing-and-limits.md
Normal file
593
references/pricing-and-limits.md
Normal file
@@ -0,0 +1,593 @@
|
||||
# Pricing and Limits Reference
|
||||
|
||||
Complete breakdown of Cloudflare Browser Rendering pricing, limits, and cost optimization strategies.
|
||||
|
||||
---
|
||||
|
||||
## Pricing Overview
|
||||
|
||||
Browser Rendering is billed on **two metrics**:
|
||||
|
||||
1. **Duration** - Total browser hours used
|
||||
2. **Concurrency** - Monthly average of concurrent browsers (Workers Bindings only)
|
||||
|
||||
---
|
||||
|
||||
## Free Tier (Workers Free Plan)
|
||||
|
||||
| Feature | Limit |
|
||||
|---------|-------|
|
||||
| **Browser Duration** | 10 minutes per day |
|
||||
| **Concurrent Browsers** | 3 per account |
|
||||
| **New Browsers per Minute** | 3 per minute |
|
||||
| **REST API Requests** | 6 per minute |
|
||||
| **Browser Timeout (Idle)** | 60 seconds |
|
||||
| **Max Session Duration** | No hard limit (closes on idle timeout) |
|
||||
|
||||
### Free Tier Use Cases
|
||||
|
||||
**Good for:**
|
||||
- Development and testing
|
||||
- Personal projects
|
||||
- Low-traffic screenshot services (<100 requests/day)
|
||||
- Learning and experimentation
|
||||
|
||||
**Not suitable for:**
|
||||
- Production applications
|
||||
- High-traffic services
|
||||
- Long-running scraping jobs
|
||||
- Batch operations (>3 concurrent browsers)
|
||||
|
||||
---
|
||||
|
||||
## Paid Tier (Workers Paid Plan)
|
||||
|
||||
### Included Limits
|
||||
|
||||
| Feature | Included |
|
||||
|---------|----------|
|
||||
| **Browser Duration** | 10 hours per month |
|
||||
| **Concurrent Browsers** | 10 (monthly average) |
|
||||
| **New Browsers per Minute** | 30 per minute |
|
||||
| **REST API Requests** | 180 per minute |
|
||||
| **Max Concurrent Browsers** | 30 per account |
|
||||
| **Browser Timeout** | 60 seconds (extendable to 10 minutes with keep_alive) |
|
||||
|
||||
### Beyond Included Limits
|
||||
|
||||
| Metric | Price |
|
||||
|--------|-------|
|
||||
| **Additional Browser Hours** | $0.09 per hour |
|
||||
| **Additional Concurrent Browsers** | $2.00 per browser (monthly average) |
|
||||
|
||||
### Requesting Higher Limits
|
||||
|
||||
If you need more than:
|
||||
- 30 concurrent browsers
|
||||
- 30 new browsers per minute
|
||||
- 180 REST API requests per minute
|
||||
|
||||
**Request higher limits**: https://forms.gle/CdueDKvb26mTaepa9
|
||||
|
||||
---
|
||||
|
||||
## Rate Limits
|
||||
|
||||
### Per-Second Enforcement
|
||||
|
||||
Rate limits are enforced **per-second**, not per-minute.
|
||||
|
||||
**Example**: 180 requests per minute = 3 requests per second
|
||||
|
||||
**This means:**
|
||||
- ❌ Cannot send all 180 requests at once
|
||||
- ✅ Must spread evenly over the minute (3/second)
|
||||
|
||||
**Implementation:**
|
||||
```typescript
|
||||
async function rateLimitedLaunch(env: Env): Promise<Browser> {
|
||||
const limits = await puppeteer.limits(env.MYBROWSER);
|
||||
|
||||
if (limits.allowedBrowserAcquisitions === 0) {
|
||||
const delay = limits.timeUntilNextAllowedBrowserAcquisition;
|
||||
await new Promise(resolve => setTimeout(resolve, delay));
|
||||
}
|
||||
|
||||
return await puppeteer.launch(env.MYBROWSER);
|
||||
}
|
||||
```
|
||||
|
||||
### Free Tier Rate Limits
|
||||
|
||||
- **Concurrent browsers**: 3
|
||||
- **New browsers/minute**: 3 (= 1 every 20 seconds)
|
||||
- **REST API requests/minute**: 6 (= 1 every 10 seconds)
|
||||
|
||||
### Paid Tier Rate Limits
|
||||
|
||||
- **Concurrent browsers**: 30 (default, can request higher)
|
||||
- **New browsers/minute**: 30 (= 1 every 2 seconds)
|
||||
- **REST API requests/minute**: 180 (= 3 per second)
|
||||
|
||||
---
|
||||
|
||||
## Duration Billing
|
||||
|
||||
### How It Works
|
||||
|
||||
1. **Daily Totals**: Cloudflare sums all browser usage each day (in seconds)
|
||||
2. **Monthly Total**: Sum of all daily totals
|
||||
3. **Rounded to Hours**: Total rounded to nearest hour
|
||||
4. **Billed**: Total hours minus 10 included hours
|
||||
|
||||
**Example:**
|
||||
- Day 1: 60 seconds (1 minute)
|
||||
- Day 2: 120 seconds (2 minutes)
|
||||
- ...
|
||||
- Day 30: 90 seconds (1.5 minutes)
|
||||
- **Monthly Total**: 45 minutes = 0.75 hours (rounded to 1 hour)
|
||||
- **Billable**: 1 hour - 10 included = 0 hours (still within free allowance)
|
||||
|
||||
### Failed Requests
|
||||
|
||||
**Failed requests are NOT billed** if they fail with `waitForTimeout` error.
|
||||
|
||||
**Example:**
|
||||
```typescript
|
||||
try {
|
||||
await page.goto(url, { timeout: 30000 });
|
||||
} catch (error) {
|
||||
// If this times out, browser time is NOT charged
|
||||
console.log("Navigation timeout - not billed");
|
||||
}
|
||||
```
|
||||
|
||||
### Duration Optimization
|
||||
|
||||
**Minimize browser time:**
|
||||
|
||||
1. **Close browsers promptly**
|
||||
```typescript
|
||||
await browser.close(); // Don't leave hanging
|
||||
```
|
||||
|
||||
2. **Use session reuse**
|
||||
```typescript
|
||||
// Reuse session instead of launching new browser
|
||||
const browser = await puppeteer.connect(env.MYBROWSER, sessionId);
|
||||
```
|
||||
|
||||
3. **Timeout management**
|
||||
```typescript
|
||||
// Set appropriate timeouts (don't wait forever)
|
||||
await page.goto(url, { timeout: 30000 });
|
||||
```
|
||||
|
||||
4. **Cache aggressively**
|
||||
```typescript
|
||||
// Cache screenshots in KV to avoid re-rendering
|
||||
const cached = await env.KV.get(url, { type: "arrayBuffer" });
|
||||
if (cached) return new Response(cached);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Concurrency Billing
|
||||
|
||||
### How It Works
|
||||
|
||||
1. **Daily Peak**: Cloudflare records highest concurrent browsers each day
|
||||
2. **Monthly Average**: Average of all daily peaks
|
||||
3. **Billed**: Average - 10 included browsers
|
||||
|
||||
**Formula:**
|
||||
```
|
||||
monthly_average = sum(daily_peaks) / days_in_month
|
||||
billable = max(0, monthly_average - 10)
|
||||
cost = billable * $2.00
|
||||
```
|
||||
|
||||
**Example:**
|
||||
- Days 1-15: 10 concurrent browsers (daily peak)
|
||||
- Days 16-30: 20 concurrent browsers (daily peak)
|
||||
- Monthly average: ((10 × 15) + (20 × 15)) / 30 = 15 browsers
|
||||
- Billable: 15 - 10 = 5 browsers
|
||||
- **Cost**: 5 × $2.00 = **$10.00**
|
||||
|
||||
### Concurrency vs Duration
|
||||
|
||||
| Scenario | Concurrency Impact | Duration Impact |
|
||||
|----------|-------------------|-----------------|
|
||||
| 1 browser for 10 hours | 1 concurrent browser | 10 browser hours |
|
||||
| 10 browsers for 1 hour | 10 concurrent browsers | 10 browser hours |
|
||||
| 100 browsers for 6 minutes | 100 concurrent browsers (!!) | 10 browser hours |
|
||||
|
||||
**Key Insight**: Short bursts of high concurrency are EXPENSIVE.
|
||||
|
||||
### Concurrency Optimization
|
||||
|
||||
**Minimize concurrent browsers:**
|
||||
|
||||
1. **Use multiple tabs**
|
||||
```typescript
|
||||
// ❌ Bad: 10 browsers
|
||||
for (const url of urls) {
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
// ...
|
||||
}
|
||||
|
||||
// ✅ Good: 1 browser, 10 tabs
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
await Promise.all(urls.map(async url => {
|
||||
const page = await browser.newPage();
|
||||
// ...
|
||||
}));
|
||||
```
|
||||
|
||||
2. **Session reuse**
|
||||
```typescript
|
||||
// Maintain pool of warm browsers
|
||||
// Reuse instead of launching new ones
|
||||
```
|
||||
|
||||
3. **Queue requests**
|
||||
```typescript
|
||||
// Limit concurrent operations
|
||||
const queue = new PQueue({ concurrency: 3 });
|
||||
await Promise.all(urls.map(url => queue.add(() => process(url))));
|
||||
```
|
||||
|
||||
4. **Incognito contexts**
|
||||
```typescript
|
||||
// Share browser, isolate sessions
|
||||
const context1 = await browser.createBrowserContext();
|
||||
const context2 = await browser.createBrowserContext();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Cost Examples
|
||||
|
||||
### Example 1: Screenshot Service
|
||||
|
||||
**Scenario:**
|
||||
- 10,000 screenshots per month
|
||||
- 3 second average per screenshot
|
||||
- No caching, no session reuse
|
||||
|
||||
**Duration:**
|
||||
- 10,000 × 3 seconds = 30,000 seconds = 8.33 hours
|
||||
- Billable: 8.33 - 10 = 0 hours (within free allowance)
|
||||
- **Duration Cost**: $0.00
|
||||
|
||||
**Concurrency:**
|
||||
- Assume 100 requests/hour during peak (9am-5pm weekdays)
|
||||
- 100 requests/hour ÷ 3600 seconds = 0.028 browsers/second
|
||||
- Peak: ~3 concurrent browsers
|
||||
- Daily peak (weekdays): 3 browsers
|
||||
- Daily peak (weekends): 1 browser
|
||||
- Monthly average: ((3 × 22) + (1 × 8)) / 30 = 2.5 browsers
|
||||
- Billable: 2.5 - 10 = 0 (within free allowance)
|
||||
- **Concurrency Cost**: $0.00
|
||||
|
||||
**Total: $0.00** (within free tier!)
|
||||
|
||||
---
|
||||
|
||||
### Example 2: Heavy Scraping
|
||||
|
||||
**Scenario:**
|
||||
- 1,000 URLs per day
|
||||
- 10 seconds average per URL
|
||||
- Batch processing (10 concurrent browsers)
|
||||
|
||||
**Duration:**
|
||||
- 1,000 × 10 seconds × 30 days = 300,000 seconds = 83.33 hours
|
||||
- Billable: 83.33 - 10 = 73.33 hours
|
||||
- **Duration Cost**: 73.33 × $0.09 = **$6.60**
|
||||
|
||||
**Concurrency:**
|
||||
- Daily peak: 10 concurrent browsers (every day)
|
||||
- Monthly average: 10 browsers
|
||||
- Billable: 10 - 10 = 0 (within free allowance)
|
||||
- **Concurrency Cost**: $0.00
|
||||
|
||||
**Total: $6.60/month**
|
||||
|
||||
---
|
||||
|
||||
### Example 3: Burst Traffic
|
||||
|
||||
**Scenario:**
|
||||
- Newsletter sent monthly with screenshot links
|
||||
- 10,000 screenshots in 1 hour
|
||||
- Each screenshot: 3 seconds
|
||||
|
||||
**Duration:**
|
||||
- 10,000 × 3 seconds = 30,000 seconds = 8.33 hours
|
||||
- Billable: 8.33 - 10 = 0 hours
|
||||
- **Duration Cost**: $0.00
|
||||
|
||||
**Concurrency:**
|
||||
- 10,000 screenshots in 1 hour = 166 requests/minute
|
||||
- At 3 seconds each: ~8.3 concurrent browsers
|
||||
- But limited to 30 max, so likely queueing
|
||||
- Daily peak: 30 browsers (rate limit)
|
||||
- Monthly average: (30 × 1 day + 1 × 29 days) / 30 = 1.97 browsers
|
||||
- Billable: 1.97 - 10 = 0
|
||||
- **Concurrency Cost**: $0.00
|
||||
|
||||
**Total: $0.00**
|
||||
|
||||
**Note**: Would hit rate limits. Better to spread over longer period or request higher limits.
|
||||
|
||||
---
|
||||
|
||||
### Example 4: Production API (Optimized)
|
||||
|
||||
**Scenario:**
|
||||
- 100,000 screenshots per month
|
||||
- Session reuse + KV caching (90% cache hit rate)
|
||||
- 10,000 actual browser renderings
|
||||
- 5 seconds average per render
|
||||
- Maintain pool of 5 warm browsers
|
||||
|
||||
**Duration:**
|
||||
- 10,000 × 5 seconds = 50,000 seconds = 13.89 hours
|
||||
- Billable: 13.89 - 10 = 3.89 hours
|
||||
- **Duration Cost**: 3.89 × $0.09 = **$0.35**
|
||||
|
||||
**Concurrency:**
|
||||
- Maintain pool of 5 browsers (keep_alive)
|
||||
- Daily peak: 5 browsers
|
||||
- Monthly average: 5 browsers
|
||||
- Billable: 5 - 10 = 0
|
||||
- **Concurrency Cost**: $0.00
|
||||
|
||||
**Total: $0.35/month** for 100k requests!
|
||||
|
||||
**ROI**: $0.0000035 per screenshot
|
||||
|
||||
---
|
||||
|
||||
## Cost Optimization Strategies
|
||||
|
||||
### 1. Aggressive Caching
|
||||
|
||||
**Strategy**: Cache screenshots/PDFs in KV or R2
|
||||
|
||||
**Impact**:
|
||||
- Reduces browser hours by 80-95%
|
||||
- Reduces concurrency needs
|
||||
- Faster response times
|
||||
|
||||
**Implementation**:
|
||||
```typescript
|
||||
// Check cache first
|
||||
const cached = await env.KV.get(url, { type: "arrayBuffer" });
|
||||
if (cached) return new Response(cached);
|
||||
|
||||
// Generate and cache
|
||||
const screenshot = await generateScreenshot(url);
|
||||
await env.KV.put(url, screenshot, { expirationTtl: 86400 });
|
||||
```
|
||||
|
||||
**Cost Savings**: 80-95% reduction
|
||||
|
||||
---
|
||||
|
||||
### 2. Session Reuse
|
||||
|
||||
**Strategy**: Maintain pool of warm browsers, reuse sessions
|
||||
|
||||
**Impact**:
|
||||
- Reduces cold start time
|
||||
- Lower concurrency charges
|
||||
- Better throughput
|
||||
|
||||
**Implementation**: See `session-reuse.ts` template
|
||||
|
||||
**Cost Savings**: 30-50% reduction
|
||||
|
||||
---
|
||||
|
||||
### 3. Multiple Tabs
|
||||
|
||||
**Strategy**: Use tabs instead of multiple browsers
|
||||
|
||||
**Impact**:
|
||||
- 10-50x reduction in concurrency
|
||||
- Minimal duration increase
|
||||
- Much cheaper
|
||||
|
||||
**Implementation**:
|
||||
```typescript
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
await Promise.all(urls.map(async url => {
|
||||
const page = await browser.newPage();
|
||||
// process
|
||||
await page.close();
|
||||
}));
|
||||
await browser.close();
|
||||
```
|
||||
|
||||
**Cost Savings**: 90%+ reduction in concurrency charges
|
||||
|
||||
---
|
||||
|
||||
### 4. Appropriate Timeouts
|
||||
|
||||
**Strategy**: Set reasonable timeouts, don't wait forever
|
||||
|
||||
**Impact**:
|
||||
- Prevents hanging browsers
|
||||
- Reduces wasted duration
|
||||
- Better error handling
|
||||
|
||||
**Implementation**:
|
||||
```typescript
|
||||
await page.goto(url, {
|
||||
timeout: 30000, // 30 second max
|
||||
waitUntil: "networkidle0"
|
||||
});
|
||||
```
|
||||
|
||||
**Cost Savings**: 20-40% reduction
|
||||
|
||||
---
|
||||
|
||||
### 5. Request Queueing
|
||||
|
||||
**Strategy**: Limit concurrent operations to stay within limits
|
||||
|
||||
**Impact**:
|
||||
- Avoid rate limit errors
|
||||
- Predictable costs
|
||||
- Better resource utilization
|
||||
|
||||
**Implementation**:
|
||||
```typescript
|
||||
import PQueue from "p-queue";
|
||||
|
||||
const queue = new PQueue({ concurrency: 5 });
|
||||
|
||||
await Promise.all(urls.map(url =>
|
||||
queue.add(() => processUrl(url))
|
||||
));
|
||||
```
|
||||
|
||||
**Cost Savings**: Avoids rate limit charges
|
||||
|
||||
---
|
||||
|
||||
## Monitoring Usage
|
||||
|
||||
### Dashboard
|
||||
|
||||
View usage in Cloudflare Dashboard:
|
||||
|
||||
https://dash.cloudflare.com/?to=/:account/workers/browser-rendering
|
||||
|
||||
**Metrics available:**
|
||||
- Total browser hours used
|
||||
- REST API requests
|
||||
- Concurrent browsers (graph)
|
||||
- Cost estimates
|
||||
|
||||
### Response Headers
|
||||
|
||||
REST API returns browser time used:
|
||||
|
||||
```
|
||||
X-Browser-Ms-Used: 2340
|
||||
```
|
||||
|
||||
(Browser time in milliseconds for that request)
|
||||
|
||||
### Custom Tracking
|
||||
|
||||
```typescript
|
||||
interface UsageMetrics {
|
||||
date: string;
|
||||
browserHours: number;
|
||||
peakConcurrency: number;
|
||||
requests: number;
|
||||
cacheHitRate: number;
|
||||
}
|
||||
|
||||
// Track in D1 or Analytics Engine
|
||||
await env.ANALYTICS.writeDataPoint({
|
||||
indexes: [date],
|
||||
blobs: ["browser_usage"],
|
||||
doubles: [browserHours, peakConcurrency, requests]
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Cost Alerts
|
||||
|
||||
### Set Up Alerts
|
||||
|
||||
1. **Monitor daily peaks**
|
||||
```typescript
|
||||
const limits = await puppeteer.limits(env.MYBROWSER);
|
||||
if (limits.activeSessions.length > 15) {
|
||||
console.warn("High concurrency detected:", limits.activeSessions.length);
|
||||
}
|
||||
```
|
||||
|
||||
2. **Track hourly usage**
|
||||
```typescript
|
||||
const usage = await getHourlyUsage();
|
||||
if (usage.browserHours > 1) {
|
||||
console.warn("High browser usage this hour:", usage.browserHours);
|
||||
}
|
||||
```
|
||||
|
||||
3. **Set budget limits**
|
||||
```typescript
|
||||
const monthlyBudget = 50; // $50/month
|
||||
const currentCost = await estimateCurrentCost();
|
||||
if (currentCost > monthlyBudget * 0.8) {
|
||||
console.warn("Approaching monthly budget:", currentCost);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Best Practices Summary
|
||||
|
||||
1. **Always cache** screenshots/PDFs in KV or R2
|
||||
2. **Reuse sessions** instead of launching new browsers
|
||||
3. **Use multiple tabs** instead of multiple browsers
|
||||
4. **Set appropriate timeouts** to prevent hanging
|
||||
5. **Monitor usage** in dashboard and logs
|
||||
6. **Queue requests** to stay within rate limits
|
||||
7. **Test caching** to optimize hit rate
|
||||
8. **Profile operations** to identify slow requests
|
||||
9. **Use incognito contexts** for session isolation
|
||||
10. **Request higher limits** if needed for production
|
||||
|
||||
---
|
||||
|
||||
## Common Questions
|
||||
|
||||
### Q: Are failed requests billed?
|
||||
|
||||
**A**: No. Requests that fail with `waitForTimeout` error are NOT billed.
|
||||
|
||||
### Q: How is concurrency calculated?
|
||||
|
||||
**A**: Monthly average of daily peak concurrent browsers.
|
||||
|
||||
### Q: Can I reduce my bill?
|
||||
|
||||
**A**: Yes! Use caching, session reuse, and multiple tabs. See optimization strategies above.
|
||||
|
||||
### Q: What if I hit limits?
|
||||
|
||||
**A**: Implement queueing, or request higher limits: https://forms.gle/CdueDKvb26mTaepa9
|
||||
|
||||
### Q: Is there a free tier?
|
||||
|
||||
**A**: Yes! 10 minutes/day browser time, 3 concurrent browsers.
|
||||
|
||||
### Q: How do I estimate costs?
|
||||
|
||||
**A**: Monitor usage in dashboard, then calculate:
|
||||
- Duration: (hours - 10) × $0.09
|
||||
- Concurrency: (avg - 10) × $2.00
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- **Official Pricing Docs**: https://developers.cloudflare.com/browser-rendering/platform/pricing/
|
||||
- **Limits Docs**: https://developers.cloudflare.com/browser-rendering/platform/limits/
|
||||
- **Dashboard**: https://dash.cloudflare.com/?to=/:account/workers/browser-rendering
|
||||
- **Request Higher Limits**: https://forms.gle/CdueDKvb26mTaepa9
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-10-22
|
||||
627
references/puppeteer-vs-playwright.md
Normal file
627
references/puppeteer-vs-playwright.md
Normal file
@@ -0,0 +1,627 @@
|
||||
# Puppeteer vs Playwright Comparison
|
||||
|
||||
Complete comparison guide for choosing between @cloudflare/puppeteer and @cloudflare/playwright.
|
||||
|
||||
---
|
||||
|
||||
## Quick Recommendation
|
||||
|
||||
**Use Puppeteer if:**
|
||||
- ✅ Starting a new project
|
||||
- ✅ Need session management features
|
||||
- ✅ Want to optimize performance/costs
|
||||
- ✅ Building screenshot/PDF services
|
||||
- ✅ Web scraping workflows
|
||||
|
||||
**Use Playwright if:**
|
||||
- ✅ Already have Playwright tests to migrate
|
||||
- ✅ Prefer auto-waiting behavior
|
||||
- ✅ Don't need advanced session features
|
||||
- ✅ Want cross-browser APIs (even if only Chromium supported now)
|
||||
|
||||
**Bottom Line**: **Puppeteer is recommended** for most Browser Rendering use cases.
|
||||
|
||||
---
|
||||
|
||||
## Package Installation
|
||||
|
||||
### Puppeteer
|
||||
```bash
|
||||
npm install @cloudflare/puppeteer
|
||||
```
|
||||
|
||||
**Version**: 1.0.4 (based on Puppeteer v23.x)
|
||||
|
||||
### Playwright
|
||||
```bash
|
||||
npm install @cloudflare/playwright
|
||||
```
|
||||
|
||||
**Version**: 1.0.0 (based on Playwright v1.55.0)
|
||||
|
||||
---
|
||||
|
||||
## API Comparison
|
||||
|
||||
### Launching a Browser
|
||||
|
||||
**Puppeteer:**
|
||||
```typescript
|
||||
import puppeteer from "@cloudflare/puppeteer";
|
||||
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
```
|
||||
|
||||
**Playwright:**
|
||||
```typescript
|
||||
import { chromium } from "@cloudflare/playwright";
|
||||
|
||||
const browser = await chromium.launch(env.BROWSER);
|
||||
```
|
||||
|
||||
**Key Difference**: Playwright uses `chromium.launch()` (browser-specific), Puppeteer uses `puppeteer.launch()` (generic).
|
||||
|
||||
---
|
||||
|
||||
### Basic Screenshot Example
|
||||
|
||||
**Puppeteer:**
|
||||
```typescript
|
||||
import puppeteer from "@cloudflare/puppeteer";
|
||||
|
||||
export default {
|
||||
async fetch(request: Request, env: Env): Promise<Response> {
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
const page = await browser.newPage();
|
||||
await page.goto("https://example.com");
|
||||
const screenshot = await page.screenshot();
|
||||
await browser.close();
|
||||
|
||||
return new Response(screenshot, {
|
||||
headers: { "content-type": "image/png" }
|
||||
});
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
**Playwright:**
|
||||
```typescript
|
||||
import { chromium } from "@cloudflare/playwright";
|
||||
|
||||
export default {
|
||||
async fetch(request: Request, env: Env): Promise<Response> {
|
||||
const browser = await chromium.launch(env.BROWSER);
|
||||
const page = await browser.newPage();
|
||||
await page.goto("https://example.com");
|
||||
const screenshot = await page.screenshot();
|
||||
await browser.close();
|
||||
|
||||
return new Response(screenshot, {
|
||||
headers: { "content-type": "image/png" }
|
||||
});
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
**Key Difference**: Nearly identical! Main difference is import and launch method.
|
||||
|
||||
---
|
||||
|
||||
## Feature Comparison
|
||||
|
||||
| Feature | Puppeteer | Playwright | Notes |
|
||||
|---------|-----------|------------|-------|
|
||||
| **Basic Screenshots** | ✅ Yes | ✅ Yes | Both support PNG/JPEG |
|
||||
| **PDF Generation** | ✅ Yes | ✅ Yes | Identical API |
|
||||
| **Page Navigation** | ✅ Yes | ✅ Yes | Similar API |
|
||||
| **Element Selectors** | CSS only | CSS, text | Playwright has more selector types |
|
||||
| **Auto-waiting** | ❌ Manual | ✅ Built-in | Playwright waits for elements automatically |
|
||||
| **Session Management** | ✅ Advanced | ⚠️ Basic | Puppeteer has .sessions(), .history(), .limits() |
|
||||
| **Session Reuse** | ✅ Yes | ⚠️ Limited | Puppeteer has .connect() with sessionId |
|
||||
| **Browser Contexts** | ✅ Yes | ✅ Yes | Both support incognito contexts |
|
||||
| **Multiple Tabs** | ✅ Yes | ✅ Yes | Both support multiple pages |
|
||||
| **Network Interception** | ✅ Yes | ✅ Yes | Similar APIs |
|
||||
| **Geolocation** | ✅ Yes | ✅ Yes | Similar APIs |
|
||||
| **Emulation** | ✅ Yes | ✅ Yes | Device emulation, viewport |
|
||||
| **Browser Support** | Chromium only | Chromium only | Firefox/Safari not yet supported |
|
||||
| **TypeScript Types** | ✅ Yes | ✅ Yes | Both fully typed |
|
||||
|
||||
---
|
||||
|
||||
## Session Management
|
||||
|
||||
### Puppeteer (Advanced)
|
||||
|
||||
```typescript
|
||||
// List active sessions
|
||||
const sessions = await puppeteer.sessions(env.MYBROWSER);
|
||||
|
||||
// Find free session
|
||||
const freeSession = sessions.find(s => !s.connectionId);
|
||||
|
||||
// Connect to existing session
|
||||
if (freeSession) {
|
||||
const browser = await puppeteer.connect(env.MYBROWSER, freeSession.sessionId);
|
||||
}
|
||||
|
||||
// Check limits
|
||||
const limits = await puppeteer.limits(env.MYBROWSER);
|
||||
console.log("Can launch:", limits.allowedBrowserAcquisitions > 0);
|
||||
|
||||
// View history
|
||||
const history = await puppeteer.history(env.MYBROWSER);
|
||||
```
|
||||
|
||||
**Puppeteer APIs:**
|
||||
- ✅ `puppeteer.sessions()` - List active sessions
|
||||
- ✅ `puppeteer.connect()` - Connect to session by ID
|
||||
- ✅ `puppeteer.history()` - View recent sessions
|
||||
- ✅ `puppeteer.limits()` - Check account limits
|
||||
- ✅ `browser.sessionId()` - Get current session ID
|
||||
- ✅ `browser.disconnect()` - Disconnect without closing
|
||||
|
||||
---
|
||||
|
||||
### Playwright (Basic)
|
||||
|
||||
```typescript
|
||||
// Launch browser
|
||||
const browser = await chromium.launch(env.BROWSER);
|
||||
|
||||
// Get session info (basic)
|
||||
// Note: No .sessions(), .history(), or .limits() APIs
|
||||
```
|
||||
|
||||
**Playwright APIs:**
|
||||
- ❌ No `chromium.sessions()` equivalent
|
||||
- ❌ No session reuse APIs
|
||||
- ❌ No limits checking
|
||||
- ❌ No session history
|
||||
|
||||
**Workaround**: Use Puppeteer-style session management via REST API (more complex).
|
||||
|
||||
---
|
||||
|
||||
## Auto-Waiting Behavior
|
||||
|
||||
### Puppeteer (Manual)
|
||||
|
||||
```typescript
|
||||
// Must explicitly wait for elements
|
||||
await page.goto("https://example.com");
|
||||
await page.waitForSelector("button#submit");
|
||||
await page.click("button#submit");
|
||||
```
|
||||
|
||||
**Pros**: Fine-grained control
|
||||
|
||||
**Cons**: More verbose, easy to forget waits
|
||||
|
||||
---
|
||||
|
||||
### Playwright (Auto-waiting)
|
||||
|
||||
```typescript
|
||||
// Automatically waits for elements
|
||||
await page.goto("https://example.com");
|
||||
await page.click("button#submit"); // Waits automatically!
|
||||
```
|
||||
|
||||
**Pros**: Less boilerplate, fewer timing issues
|
||||
|
||||
**Cons**: Less control over wait behavior
|
||||
|
||||
---
|
||||
|
||||
## Selector Support
|
||||
|
||||
### Puppeteer
|
||||
|
||||
**Supported:**
|
||||
- CSS selectors: `"button#submit"`, `"div > p"`
|
||||
- `:visible`, `:hidden` pseudo-classes
|
||||
- `page.$()`, `page.$$()` for querying
|
||||
|
||||
**Not Supported:**
|
||||
- XPath selectors (use `page.evaluate()` workaround)
|
||||
- Text selectors
|
||||
- Layout selectors
|
||||
|
||||
**Example:**
|
||||
```typescript
|
||||
// CSS selector
|
||||
const button = await page.$("button#submit");
|
||||
|
||||
// XPath workaround
|
||||
const heading = await page.evaluate(() => {
|
||||
return new XPathEvaluator()
|
||||
.createExpression("//h1[@class='title']")
|
||||
.evaluate(document, XPathResult.FIRST_ORDERED_NODE_TYPE)
|
||||
.singleNodeValue.textContent;
|
||||
});
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Playwright
|
||||
|
||||
**Supported:**
|
||||
- CSS selectors: `"button#submit"`
|
||||
- Text selectors: `"text=Submit"`
|
||||
- XPath selectors: `"xpath=//button"`
|
||||
- Layout selectors: `"button :right-of(:text('Cancel'))"`
|
||||
|
||||
**Example:**
|
||||
```typescript
|
||||
// CSS selector
|
||||
await page.click("button#submit");
|
||||
|
||||
// Text selector
|
||||
await page.click("text=Submit");
|
||||
|
||||
// Combined selector
|
||||
await page.click("button >> text=Submit");
|
||||
```
|
||||
|
||||
**Advantage**: More flexible selector options
|
||||
|
||||
---
|
||||
|
||||
## Performance & Cost
|
||||
|
||||
### Puppeteer (Optimized)
|
||||
|
||||
**Session Reuse:**
|
||||
```typescript
|
||||
// Reuse sessions to reduce costs
|
||||
const sessions = await puppeteer.sessions(env.MYBROWSER);
|
||||
const browser = await puppeteer.connect(env.MYBROWSER, sessionId);
|
||||
await browser.disconnect(); // Keep alive
|
||||
```
|
||||
|
||||
**Cost Impact:**
|
||||
- ✅ Reduce cold starts by 50-70%
|
||||
- ✅ Lower concurrency charges
|
||||
- ✅ Better throughput
|
||||
|
||||
---
|
||||
|
||||
### Playwright (Limited Optimization)
|
||||
|
||||
**No Session Reuse:**
|
||||
```typescript
|
||||
// Must launch new browser each time
|
||||
const browser = await chromium.launch(env.BROWSER);
|
||||
await browser.close(); // Cannot keep alive for reuse
|
||||
```
|
||||
|
||||
**Cost Impact:**
|
||||
- ❌ Higher browser hours (cold starts every request)
|
||||
- ❌ Higher concurrency usage
|
||||
- ❌ Lower throughput
|
||||
|
||||
**Difference**: ~30-50% higher costs with Playwright vs optimized Puppeteer.
|
||||
|
||||
---
|
||||
|
||||
## API Differences
|
||||
|
||||
| Operation | Puppeteer | Playwright |
|
||||
|-----------|-----------|------------|
|
||||
| **Import** | `import puppeteer from "@cloudflare/puppeteer"` | `import { chromium } from "@cloudflare/playwright"` |
|
||||
| **Launch** | `puppeteer.launch(env.MYBROWSER)` | `chromium.launch(env.BROWSER)` |
|
||||
| **Connect** | `puppeteer.connect(env.MYBROWSER, sessionId)` | ❌ Not available |
|
||||
| **Sessions** | `puppeteer.sessions(env.MYBROWSER)` | ❌ Not available |
|
||||
| **Limits** | `puppeteer.limits(env.MYBROWSER)` | ❌ Not available |
|
||||
| **Goto** | `page.goto(url, { waitUntil: "networkidle0" })` | `page.goto(url, { waitUntil: "networkidle" })` |
|
||||
| **Screenshot** | `page.screenshot({ fullPage: true })` | `page.screenshot({ fullPage: true })` |
|
||||
| **PDF** | `page.pdf({ format: "A4" })` | `page.pdf({ format: "A4" })` |
|
||||
| **Wait** | `page.waitForSelector("button")` | `page.locator("button").waitFor()` |
|
||||
| **Click** | `page.click("button")` | `page.click("button")` (auto-waits) |
|
||||
|
||||
---
|
||||
|
||||
## Migration Guide
|
||||
|
||||
### Puppeteer → Playwright
|
||||
|
||||
```typescript
|
||||
// Before (Puppeteer)
|
||||
import puppeteer from "@cloudflare/puppeteer";
|
||||
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
const page = await browser.newPage();
|
||||
await page.goto(url, { waitUntil: "networkidle0" });
|
||||
await page.waitForSelector("button#submit");
|
||||
await page.click("button#submit");
|
||||
const screenshot = await page.screenshot();
|
||||
await browser.close();
|
||||
```
|
||||
|
||||
```typescript
|
||||
// After (Playwright)
|
||||
import { chromium } from "@cloudflare/playwright";
|
||||
|
||||
const browser = await chromium.launch(env.BROWSER);
|
||||
const page = await browser.newPage();
|
||||
await page.goto(url, { waitUntil: "networkidle" });
|
||||
// No waitForSelector needed - auto-waits
|
||||
await page.click("button#submit");
|
||||
const screenshot = await page.screenshot();
|
||||
await browser.close();
|
||||
```
|
||||
|
||||
**Changes:**
|
||||
1. Import: `puppeteer` → `{ chromium }`
|
||||
2. Launch: `puppeteer.launch()` → `chromium.launch()`
|
||||
3. Wait: `networkidle0` → `networkidle`
|
||||
4. Remove explicit `waitForSelector()` (auto-waits)
|
||||
|
||||
---
|
||||
|
||||
### Playwright → Puppeteer
|
||||
|
||||
```typescript
|
||||
// Before (Playwright)
|
||||
import { chromium } from "@cloudflare/playwright";
|
||||
|
||||
const browser = await chromium.launch(env.BROWSER);
|
||||
const page = await browser.newPage();
|
||||
await page.goto(url);
|
||||
await page.click("button#submit"); // Auto-waits
|
||||
```
|
||||
|
||||
```typescript
|
||||
// After (Puppeteer)
|
||||
import puppeteer from "@cloudflare/puppeteer";
|
||||
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
const page = await browser.newPage();
|
||||
await page.goto(url, { waitUntil: "networkidle0" });
|
||||
await page.waitForSelector("button#submit"); // Explicit wait
|
||||
await page.click("button#submit");
|
||||
```
|
||||
|
||||
**Changes:**
|
||||
1. Import: `{ chromium }` → `puppeteer`
|
||||
2. Launch: `chromium.launch()` → `puppeteer.launch()`
|
||||
3. Add explicit waits: `page.waitForSelector()`
|
||||
4. Specify wait conditions: `waitUntil: "networkidle0"`
|
||||
|
||||
---
|
||||
|
||||
## Use Case Recommendations
|
||||
|
||||
### Screenshot Service
|
||||
**Winner**: **Puppeteer**
|
||||
|
||||
**Reason**: Session reuse reduces costs by 30-50%
|
||||
|
||||
```typescript
|
||||
// Puppeteer: Reuse sessions
|
||||
const sessions = await puppeteer.sessions(env.MYBROWSER);
|
||||
const browser = await puppeteer.connect(env.MYBROWSER, sessionId);
|
||||
await browser.disconnect(); // Keep alive
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### PDF Generation
|
||||
**Winner**: **Tie**
|
||||
|
||||
**Reason**: Identical API, no session reuse benefit
|
||||
|
||||
```typescript
|
||||
// Both have same API
|
||||
const pdf = await page.pdf({ format: "A4" });
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Web Scraping
|
||||
**Winner**: **Puppeteer**
|
||||
|
||||
**Reason**: Session management + limit checking
|
||||
|
||||
```typescript
|
||||
// Puppeteer: Check limits before scraping
|
||||
const limits = await puppeteer.limits(env.MYBROWSER);
|
||||
if (limits.allowedBrowserAcquisitions === 0) {
|
||||
await delay(limits.timeUntilNextAllowedBrowserAcquisition);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Test Migration
|
||||
**Winner**: **Playwright**
|
||||
|
||||
**Reason**: Easier to migrate existing Playwright tests
|
||||
|
||||
```typescript
|
||||
// Minimal changes needed
|
||||
// Just update imports and launch
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Interactive Automation
|
||||
**Winner**: **Tie**
|
||||
|
||||
**Reason**: Both support form filling, clicking, etc.
|
||||
|
||||
---
|
||||
|
||||
## Configuration
|
||||
|
||||
### wrangler.jsonc (Puppeteer)
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"browser": {
|
||||
"binding": "MYBROWSER"
|
||||
},
|
||||
"compatibility_flags": ["nodejs_compat"]
|
||||
}
|
||||
```
|
||||
|
||||
```typescript
|
||||
interface Env {
|
||||
MYBROWSER: Fetcher;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### wrangler.jsonc (Playwright)
|
||||
|
||||
```jsonc
|
||||
{
|
||||
"browser": {
|
||||
"binding": "BROWSER"
|
||||
},
|
||||
"compatibility_flags": ["nodejs_compat"]
|
||||
}
|
||||
```
|
||||
|
||||
```typescript
|
||||
interface Env {
|
||||
BROWSER: Fetcher;
|
||||
}
|
||||
```
|
||||
|
||||
**Note**: Binding name is arbitrary, but convention is `MYBROWSER` for Puppeteer and `BROWSER` for Playwright.
|
||||
|
||||
---
|
||||
|
||||
## Production Considerations
|
||||
|
||||
### Puppeteer Advantages
|
||||
- ✅ Session reuse (30-50% cost savings)
|
||||
- ✅ Limit checking (`puppeteer.limits()`)
|
||||
- ✅ Session monitoring (`puppeteer.sessions()`, `.history()`)
|
||||
- ✅ Better performance optimization options
|
||||
- ✅ More mature Cloudflare fork
|
||||
|
||||
### Playwright Advantages
|
||||
- ✅ Auto-waiting (less code)
|
||||
- ✅ More selector types
|
||||
- ✅ Better cross-browser APIs (future-proof)
|
||||
- ✅ Easier migration from existing tests
|
||||
|
||||
---
|
||||
|
||||
## Recommendation Summary
|
||||
|
||||
| Scenario | Recommended | Reason |
|
||||
|----------|-------------|--------|
|
||||
| New project | **Puppeteer** | Session management + cost optimization |
|
||||
| Screenshot service | **Puppeteer** | Session reuse saves 30-50% |
|
||||
| PDF generation | **Tie** | Identical API |
|
||||
| Web scraping | **Puppeteer** | Limit checking + session management |
|
||||
| Migrating Playwright tests | **Playwright** | Minimal changes needed |
|
||||
| High traffic production | **Puppeteer** | Better performance optimization |
|
||||
| Quick prototype | **Tie** | Both easy to start with |
|
||||
|
||||
---
|
||||
|
||||
## Code Examples
|
||||
|
||||
### Puppeteer (Production-Optimized)
|
||||
|
||||
```typescript
|
||||
import puppeteer, { Browser } from "@cloudflare/puppeteer";
|
||||
|
||||
async function getBrowser(env: Env): Promise<Browser> {
|
||||
// Check limits
|
||||
const limits = await puppeteer.limits(env.MYBROWSER);
|
||||
if (limits.allowedBrowserAcquisitions === 0) {
|
||||
throw new Error("Rate limit reached");
|
||||
}
|
||||
|
||||
// Try to reuse session
|
||||
const sessions = await puppeteer.sessions(env.MYBROWSER);
|
||||
const freeSession = sessions.find(s => !s.connectionId);
|
||||
|
||||
if (freeSession) {
|
||||
try {
|
||||
return await puppeteer.connect(env.MYBROWSER, freeSession.sessionId);
|
||||
} catch {
|
||||
// Session closed, launch new
|
||||
}
|
||||
}
|
||||
|
||||
return await puppeteer.launch(env.MYBROWSER);
|
||||
}
|
||||
|
||||
export default {
|
||||
async fetch(request: Request, env: Env): Promise<Response> {
|
||||
const browser = await getBrowser(env);
|
||||
|
||||
try {
|
||||
const page = await browser.newPage();
|
||||
await page.goto("https://example.com", {
|
||||
waitUntil: "networkidle0",
|
||||
timeout: 30000
|
||||
});
|
||||
const screenshot = await page.screenshot();
|
||||
|
||||
// Disconnect (keep alive)
|
||||
await browser.disconnect();
|
||||
|
||||
return new Response(screenshot, {
|
||||
headers: { "content-type": "image/png" }
|
||||
});
|
||||
} catch (error) {
|
||||
await browser.close();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Playwright (Simple)
|
||||
|
||||
```typescript
|
||||
import { chromium } from "@cloudflare/playwright";
|
||||
|
||||
export default {
|
||||
async fetch(request: Request, env: Env): Promise<Response> {
|
||||
const browser = await chromium.launch(env.BROWSER);
|
||||
|
||||
try {
|
||||
const page = await browser.newPage();
|
||||
await page.goto("https://example.com", {
|
||||
waitUntil: "networkidle",
|
||||
timeout: 30000
|
||||
});
|
||||
const screenshot = await page.screenshot();
|
||||
|
||||
await browser.close();
|
||||
|
||||
return new Response(screenshot, {
|
||||
headers: { "content-type": "image/png" }
|
||||
});
|
||||
} catch (error) {
|
||||
await browser.close();
|
||||
throw error;
|
||||
}
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## References
|
||||
|
||||
- **Puppeteer Docs**: https://pptr.dev/
|
||||
- **Playwright Docs**: https://playwright.dev/
|
||||
- **Cloudflare Puppeteer Fork**: https://github.com/cloudflare/puppeteer
|
||||
- **Cloudflare Playwright Fork**: https://github.com/cloudflare/playwright
|
||||
- **Browser Rendering Docs**: https://developers.cloudflare.com/browser-rendering/
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-10-22
|
||||
739
references/session-management.md
Normal file
739
references/session-management.md
Normal file
@@ -0,0 +1,739 @@
|
||||
# Session Management Guide
|
||||
|
||||
Complete guide to browser session management for performance optimization and concurrency handling.
|
||||
|
||||
---
|
||||
|
||||
## Why Session Management Matters
|
||||
|
||||
**The Problem:**
|
||||
- Launching new browsers is slow (~2-3 seconds cold start)
|
||||
- Each launch consumes concurrency quota
|
||||
- Free tier: Only 3 concurrent browsers
|
||||
- Paid tier: 10-30 concurrent browsers (costs $2/browser beyond included)
|
||||
|
||||
**The Solution:**
|
||||
- Reuse browser sessions across requests
|
||||
- Use multiple tabs instead of multiple browsers
|
||||
- Check limits before launching
|
||||
- Disconnect (don't close) to keep sessions alive
|
||||
|
||||
**Benefits:**
|
||||
- ⚡ **50-70% faster** (no cold start)
|
||||
- 💰 **Lower costs** (reduced concurrency charges)
|
||||
- 📊 **Better utilization** (one browser, many tabs)
|
||||
|
||||
---
|
||||
|
||||
## Session Lifecycle
|
||||
|
||||
```
|
||||
1. Launch → Browser session created (session ID assigned)
|
||||
2. Connected → Worker actively using browser
|
||||
3. Disconnected → Session idle, available for reuse
|
||||
4. Timeout → Session closed after 60s idle (configurable)
|
||||
5. Closed → Session terminated (must launch new one)
|
||||
```
|
||||
|
||||
### Session States
|
||||
|
||||
| State | Description | Can Connect? |
|
||||
|-------|-------------|--------------|
|
||||
| **Active with connection** | Worker is using browser | ❌ No (occupied) |
|
||||
| **Active without connection** | Browser idle, waiting | ✅ Yes (available) |
|
||||
| **Closed** | Session terminated | ❌ No (gone) |
|
||||
|
||||
---
|
||||
|
||||
## Session Management API
|
||||
|
||||
### puppeteer.sessions()
|
||||
|
||||
List all currently running browser sessions.
|
||||
|
||||
**Signature:**
|
||||
```typescript
|
||||
await puppeteer.sessions(binding: Fetcher): Promise<SessionInfo[]>
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```typescript
|
||||
interface SessionInfo {
|
||||
sessionId: string; // Unique session ID
|
||||
startTime: number; // Unix timestamp (ms)
|
||||
connectionId?: string; // Present if worker is connected
|
||||
connectionStartTime?: number;
|
||||
}
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```typescript
|
||||
const sessions = await puppeteer.sessions(env.MYBROWSER);
|
||||
|
||||
// Find free sessions (no active connection)
|
||||
const freeSessions = sessions.filter(s => !s.connectionId);
|
||||
|
||||
// Find occupied sessions
|
||||
const occupiedSessions = sessions.filter(s => s.connectionId);
|
||||
|
||||
console.log({
|
||||
total: sessions.length,
|
||||
available: freeSessions.length,
|
||||
occupied: occupiedSessions.length
|
||||
});
|
||||
```
|
||||
|
||||
**Output:**
|
||||
```json
|
||||
[
|
||||
{
|
||||
"sessionId": "478f4d7d-e943-40f6-a414-837d3736a1dc",
|
||||
"startTime": 1711621703708,
|
||||
"connectionId": "2a2246fa-e234-4dc1-8433-87e6cee80145",
|
||||
"connectionStartTime": 1711621704607
|
||||
},
|
||||
{
|
||||
"sessionId": "565e05fb-4d2a-402b-869b-5b65b1381db7",
|
||||
"startTime": 1711621703808
|
||||
}
|
||||
]
|
||||
```
|
||||
|
||||
**Interpretation:**
|
||||
- Session `478f4d...` is **occupied** (has connectionId)
|
||||
- Session `565e05...` is **available** (no connectionId)
|
||||
|
||||
---
|
||||
|
||||
### puppeteer.history()
|
||||
|
||||
List recent sessions, both open and closed.
|
||||
|
||||
**Signature:**
|
||||
```typescript
|
||||
await puppeteer.history(binding: Fetcher): Promise<HistoryEntry[]>
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```typescript
|
||||
interface HistoryEntry {
|
||||
sessionId: string;
|
||||
startTime: number;
|
||||
endTime?: number; // Present if closed
|
||||
closeReason?: number; // Numeric close code
|
||||
closeReasonText?: string; // Human-readable reason
|
||||
}
|
||||
```
|
||||
|
||||
**Close Reasons:**
|
||||
- `"NormalClosure"` - Explicitly closed with browser.close()
|
||||
- `"BrowserIdle"` - Timeout due to 60s idle period
|
||||
- `"Unknown"` - Unexpected closure
|
||||
|
||||
**Example:**
|
||||
```typescript
|
||||
const history = await puppeteer.history(env.MYBROWSER);
|
||||
|
||||
history.forEach(entry => {
|
||||
const duration = entry.endTime
|
||||
? (entry.endTime - entry.startTime) / 1000
|
||||
: 'still running';
|
||||
|
||||
console.log({
|
||||
sessionId: entry.sessionId,
|
||||
duration: `${duration}s`,
|
||||
closeReason: entry.closeReasonText || 'N/A'
|
||||
});
|
||||
});
|
||||
```
|
||||
|
||||
**Use Cases:**
|
||||
- Monitor browser usage patterns
|
||||
- Debug unexpected closures
|
||||
- Track session lifetimes
|
||||
- Estimate costs
|
||||
|
||||
---
|
||||
|
||||
### puppeteer.limits()
|
||||
|
||||
Check current account limits and session availability.
|
||||
|
||||
**Signature:**
|
||||
```typescript
|
||||
await puppeteer.limits(binding: Fetcher): Promise<LimitsInfo>
|
||||
```
|
||||
|
||||
**Response:**
|
||||
```typescript
|
||||
interface LimitsInfo {
|
||||
activeSessions: Array<{ id: string }>;
|
||||
maxConcurrentSessions: number;
|
||||
allowedBrowserAcquisitions: number; // Can launch now?
|
||||
timeUntilNextAllowedBrowserAcquisition: number; // ms to wait
|
||||
}
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```typescript
|
||||
const limits = await puppeteer.limits(env.MYBROWSER);
|
||||
|
||||
console.log({
|
||||
active: limits.activeSessions.length,
|
||||
max: limits.maxConcurrentSessions,
|
||||
canLaunch: limits.allowedBrowserAcquisitions > 0,
|
||||
waitTime: limits.timeUntilNextAllowedBrowserAcquisition
|
||||
});
|
||||
```
|
||||
|
||||
**Output:**
|
||||
```json
|
||||
{
|
||||
"activeSessions": [
|
||||
{ "id": "478f4d7d-e943-40f6-a414-837d3736a1dc" },
|
||||
{ "id": "565e05fb-4d2a-402b-869b-5b65b1381db7" }
|
||||
],
|
||||
"allowedBrowserAcquisitions": 1,
|
||||
"maxConcurrentSessions": 10,
|
||||
"timeUntilNextAllowedBrowserAcquisition": 0
|
||||
}
|
||||
```
|
||||
|
||||
**Interpretation:**
|
||||
- 2 sessions currently active
|
||||
- Maximum 10 concurrent sessions allowed
|
||||
- Can launch 1 more browser now
|
||||
- No wait time required
|
||||
|
||||
---
|
||||
|
||||
### puppeteer.connect()
|
||||
|
||||
Connect to an existing browser session.
|
||||
|
||||
**Signature:**
|
||||
```typescript
|
||||
await puppeteer.connect(binding: Fetcher, sessionId: string): Promise<Browser>
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```typescript
|
||||
const sessions = await puppeteer.sessions(env.MYBROWSER);
|
||||
const freeSession = sessions.find(s => !s.connectionId);
|
||||
|
||||
if (freeSession) {
|
||||
try {
|
||||
const browser = await puppeteer.connect(env.MYBROWSER, freeSession.sessionId);
|
||||
console.log("Connected to existing session:", browser.sessionId());
|
||||
} catch (error) {
|
||||
console.log("Connection failed, session may have closed");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Error Handling:**
|
||||
Session may close between `.sessions()` call and `.connect()` call. Always wrap in try-catch.
|
||||
|
||||
---
|
||||
|
||||
### browser.sessionId()
|
||||
|
||||
Get the current browser's session ID.
|
||||
|
||||
**Signature:**
|
||||
```typescript
|
||||
browser.sessionId(): string
|
||||
```
|
||||
|
||||
**Example:**
|
||||
```typescript
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
const sessionId = browser.sessionId();
|
||||
console.log("Current session:", sessionId);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### browser.disconnect()
|
||||
|
||||
Disconnect from browser WITHOUT closing it.
|
||||
|
||||
**Signature:**
|
||||
```typescript
|
||||
await browser.disconnect(): Promise<void>
|
||||
```
|
||||
|
||||
**When to use:**
|
||||
- Want to reuse session later
|
||||
- Keep browser warm for next request
|
||||
- Reduce cold start times
|
||||
|
||||
**Example:**
|
||||
```typescript
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
const sessionId = browser.sessionId();
|
||||
|
||||
// Do work
|
||||
const page = await browser.newPage();
|
||||
await page.goto("https://example.com");
|
||||
|
||||
// Disconnect (keep alive)
|
||||
await browser.disconnect();
|
||||
|
||||
// Later: reconnect
|
||||
const browserAgain = await puppeteer.connect(env.MYBROWSER, sessionId);
|
||||
```
|
||||
|
||||
**Important:**
|
||||
- Browser will still timeout after 60s idle (use `keep_alive` to extend)
|
||||
- Session remains in your concurrent browser count
|
||||
- Other workers CAN connect to this session
|
||||
|
||||
---
|
||||
|
||||
### browser.close()
|
||||
|
||||
Close the browser and terminate the session.
|
||||
|
||||
**Signature:**
|
||||
```typescript
|
||||
await browser.close(): Promise<void>
|
||||
```
|
||||
|
||||
**When to use:**
|
||||
- Done with browser completely
|
||||
- Want to free concurrency slot
|
||||
- Error occurred during processing
|
||||
|
||||
**Example:**
|
||||
```typescript
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
|
||||
try {
|
||||
// Do work
|
||||
} catch (error) {
|
||||
await browser.close(); // Clean up on error
|
||||
throw error;
|
||||
}
|
||||
|
||||
await browser.close(); // Normal cleanup
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Session Reuse Patterns
|
||||
|
||||
### Pattern 1: Simple Reuse
|
||||
|
||||
```typescript
|
||||
async function getBrowser(env: Env): Promise<Browser> {
|
||||
// Try to connect to existing session
|
||||
const sessions = await puppeteer.sessions(env.MYBROWSER);
|
||||
const freeSession = sessions.find(s => !s.connectionId);
|
||||
|
||||
if (freeSession) {
|
||||
try {
|
||||
return await puppeteer.connect(env.MYBROWSER, freeSession.sessionId);
|
||||
} catch {
|
||||
// Session closed, launch new one
|
||||
}
|
||||
}
|
||||
|
||||
// Launch new browser
|
||||
return await puppeteer.launch(env.MYBROWSER);
|
||||
}
|
||||
|
||||
export default {
|
||||
async fetch(request: Request, env: Env): Promise<Response> {
|
||||
const browser = await getBrowser(env);
|
||||
|
||||
// Do work
|
||||
const page = await browser.newPage();
|
||||
// ...
|
||||
|
||||
// Disconnect (keep alive)
|
||||
await browser.disconnect();
|
||||
|
||||
return response;
|
||||
}
|
||||
};
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Pattern 2: Reuse with Limits Check
|
||||
|
||||
```typescript
|
||||
async function getBrowserSafe(env: Env): Promise<Browser> {
|
||||
const sessions = await puppeteer.sessions(env.MYBROWSER);
|
||||
const freeSession = sessions.find(s => !s.connectionId);
|
||||
|
||||
if (freeSession) {
|
||||
try {
|
||||
return await puppeteer.connect(env.MYBROWSER, freeSession.sessionId);
|
||||
} catch {
|
||||
// Continue to launch
|
||||
}
|
||||
}
|
||||
|
||||
// Check limits before launching
|
||||
const limits = await puppeteer.limits(env.MYBROWSER);
|
||||
|
||||
if (limits.allowedBrowserAcquisitions === 0) {
|
||||
throw new Error(
|
||||
`Rate limit reached. Retry after ${limits.timeUntilNextAllowedBrowserAcquisition}ms`
|
||||
);
|
||||
}
|
||||
|
||||
return await puppeteer.launch(env.MYBROWSER);
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Pattern 3: Retry with Backoff
|
||||
|
||||
```typescript
|
||||
async function getBrowserWithRetry(
|
||||
env: Env,
|
||||
maxRetries = 3
|
||||
): Promise<Browser> {
|
||||
for (let i = 0; i < maxRetries; i++) {
|
||||
try {
|
||||
// Try existing session first
|
||||
const sessions = await puppeteer.sessions(env.MYBROWSER);
|
||||
const freeSession = sessions.find(s => !s.connectionId);
|
||||
|
||||
if (freeSession) {
|
||||
try {
|
||||
return await puppeteer.connect(env.MYBROWSER, freeSession.sessionId);
|
||||
} catch {
|
||||
// Continue to launch
|
||||
}
|
||||
}
|
||||
|
||||
// Check limits
|
||||
const limits = await puppeteer.limits(env.MYBROWSER);
|
||||
|
||||
if (limits.allowedBrowserAcquisitions > 0) {
|
||||
return await puppeteer.launch(env.MYBROWSER);
|
||||
}
|
||||
|
||||
// Rate limited, wait and retry
|
||||
if (i < maxRetries - 1) {
|
||||
const delay = Math.min(
|
||||
limits.timeUntilNextAllowedBrowserAcquisition,
|
||||
Math.pow(2, i) * 1000 // Exponential backoff
|
||||
);
|
||||
await new Promise(resolve => setTimeout(resolve, delay));
|
||||
}
|
||||
} catch (error) {
|
||||
if (i === maxRetries - 1) throw error;
|
||||
}
|
||||
}
|
||||
|
||||
throw new Error("Failed to acquire browser after retries");
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Browser Timeout Management
|
||||
|
||||
### Default Timeout
|
||||
|
||||
Browsers close after **60 seconds of inactivity** (no devtools commands).
|
||||
|
||||
**Inactivity means:**
|
||||
- No `page.goto()`
|
||||
- No `page.screenshot()`
|
||||
- No `page.evaluate()`
|
||||
- No other browser/page operations
|
||||
|
||||
### Extending Timeout with keep_alive
|
||||
|
||||
```typescript
|
||||
const browser = await puppeteer.launch(env.MYBROWSER, {
|
||||
keep_alive: 300000 // 5 minutes = 300,000 ms
|
||||
});
|
||||
```
|
||||
|
||||
**Maximum:** 600,000ms (10 minutes)
|
||||
|
||||
**Use Cases:**
|
||||
- Long-running scraping workflows
|
||||
- Multi-step form automation
|
||||
- Session reuse across multiple requests
|
||||
|
||||
**Cost Impact:**
|
||||
- Longer keep_alive = more browser hours billed
|
||||
- Only extend if actually needed
|
||||
|
||||
---
|
||||
|
||||
## Incognito Browser Contexts
|
||||
|
||||
Use browser contexts to isolate cookies/cache while sharing a browser.
|
||||
|
||||
**Benefits:**
|
||||
- 1 concurrent browser instead of N
|
||||
- Separate cookies/cache per context
|
||||
- Test multi-user scenarios
|
||||
- Session isolation
|
||||
|
||||
**Example:**
|
||||
```typescript
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
|
||||
// Create isolated contexts
|
||||
const context1 = await browser.createBrowserContext();
|
||||
const context2 = await browser.createBrowserContext();
|
||||
|
||||
// Each context has separate state
|
||||
const page1 = await context1.newPage();
|
||||
const page2 = await context2.newPage();
|
||||
|
||||
await page1.goto("https://app.example.com"); // User 1
|
||||
await page2.goto("https://app.example.com"); // User 2
|
||||
|
||||
// page1 and page2 have separate cookies
|
||||
await context1.close();
|
||||
await context2.close();
|
||||
await browser.close();
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Multiple Tabs vs Multiple Browsers
|
||||
|
||||
### ❌ Bad: Multiple Browsers
|
||||
|
||||
```typescript
|
||||
// Uses 10 concurrent browsers!
|
||||
for (const url of urls) {
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
const page = await browser.newPage();
|
||||
await page.goto(url);
|
||||
await browser.close();
|
||||
}
|
||||
```
|
||||
|
||||
**Problems:**
|
||||
- 10x concurrency usage
|
||||
- 10x cold start delays
|
||||
- May hit concurrency limits
|
||||
|
||||
---
|
||||
|
||||
### ✅ Good: Multiple Tabs
|
||||
|
||||
```typescript
|
||||
// Uses 1 concurrent browser
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
|
||||
const results = await Promise.all(
|
||||
urls.map(async (url) => {
|
||||
const page = await browser.newPage();
|
||||
await page.goto(url);
|
||||
const data = await page.evaluate(() => ({
|
||||
title: document.title
|
||||
}));
|
||||
await page.close();
|
||||
return data;
|
||||
})
|
||||
);
|
||||
|
||||
await browser.close();
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- 1 concurrent browser (10x reduction)
|
||||
- Faster (no repeated cold starts)
|
||||
- Cheaper (reduced concurrency charges)
|
||||
|
||||
---
|
||||
|
||||
## Monitoring and Debugging
|
||||
|
||||
### Log Session Activity
|
||||
|
||||
```typescript
|
||||
const browser = await puppeteer.launch(env.MYBROWSER);
|
||||
const sessionId = browser.sessionId();
|
||||
|
||||
console.log({
|
||||
event: "browser_launched",
|
||||
sessionId,
|
||||
timestamp: Date.now()
|
||||
});
|
||||
|
||||
// Do work
|
||||
|
||||
await browser.disconnect();
|
||||
|
||||
console.log({
|
||||
event: "browser_disconnected",
|
||||
sessionId,
|
||||
timestamp: Date.now()
|
||||
});
|
||||
```
|
||||
|
||||
### Track Session Metrics
|
||||
|
||||
```typescript
|
||||
interface SessionMetrics {
|
||||
sessionId: string;
|
||||
launched: boolean; // true = new, false = reused
|
||||
duration: number; // ms
|
||||
operations: number; // page navigations
|
||||
}
|
||||
|
||||
async function trackSession(env: Env, fn: (browser: Browser) => Promise<void>) {
|
||||
const start = Date.now();
|
||||
const sessions = await puppeteer.sessions(env.MYBROWSER);
|
||||
const freeSession = sessions.find(s => !s.connectionId);
|
||||
|
||||
let browser: Browser;
|
||||
let launched: boolean;
|
||||
|
||||
if (freeSession) {
|
||||
browser = await puppeteer.connect(env.MYBROWSER, freeSession.sessionId);
|
||||
launched = false;
|
||||
} else {
|
||||
browser = await puppeteer.launch(env.MYBROWSER);
|
||||
launched = true;
|
||||
}
|
||||
|
||||
await fn(browser);
|
||||
|
||||
const metrics: SessionMetrics = {
|
||||
sessionId: browser.sessionId(),
|
||||
launched,
|
||||
duration: Date.now() - start,
|
||||
operations: 1 // Track actual operations in production
|
||||
};
|
||||
|
||||
await browser.disconnect();
|
||||
|
||||
return metrics;
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Production Best Practices
|
||||
|
||||
1. **Always Check Limits**
|
||||
- Call `puppeteer.limits()` before launching
|
||||
- Handle rate limit errors gracefully
|
||||
- Implement retry with backoff
|
||||
|
||||
2. **Prefer Session Reuse**
|
||||
- Try `puppeteer.connect()` first
|
||||
- Fall back to `puppeteer.launch()` only if needed
|
||||
- Use `browser.disconnect()` instead of `browser.close()`
|
||||
|
||||
3. **Use Multiple Tabs**
|
||||
- One browser, many tabs
|
||||
- Reduces concurrency usage 10-50x
|
||||
- Faster than multiple browsers
|
||||
|
||||
4. **Set Appropriate Timeouts**
|
||||
- Default 60s is fine for most use cases
|
||||
- Extend only if actually needed (keep_alive)
|
||||
- Remember: longer timeout = more billable hours
|
||||
|
||||
5. **Handle Errors**
|
||||
- Always `browser.close()` on errors
|
||||
- Wrap `puppeteer.connect()` in try-catch
|
||||
- Gracefully handle rate limits
|
||||
|
||||
6. **Monitor Usage**
|
||||
- Log session IDs
|
||||
- Track reuse rate
|
||||
- Monitor concurrency in dashboard
|
||||
|
||||
7. **Use Incognito Contexts**
|
||||
- Isolate sessions while sharing browser
|
||||
- Better than multiple browsers
|
||||
- Test multi-user scenarios safely
|
||||
|
||||
---
|
||||
|
||||
## Cost Optimization
|
||||
|
||||
### Scenario: Screenshot Service (1000 requests/hour)
|
||||
|
||||
**Bad Approach (No Session Reuse):**
|
||||
- Launch new browser for each request
|
||||
- 1000 browsers/hour
|
||||
- Average session: 5 seconds
|
||||
- Browser hours: (1000 * 5) / 3600 = 1.39 hours
|
||||
- Average concurrency: ~14 browsers
|
||||
- **Cost**: 1.39 hours = $0.13 + (14-10) * $2 = $8.13/hour
|
||||
|
||||
**Good Approach (Session Reuse):**
|
||||
- Maintain pool of 3-5 warm browsers
|
||||
- Reuse sessions across requests
|
||||
- Average session: 1 hour (keep_alive)
|
||||
- Browser hours: 5 hours (5 browsers * 1 hour)
|
||||
- Average concurrency: 5 browsers
|
||||
- **Cost**: 5 hours = $0.45/hour
|
||||
|
||||
**Savings: 94%** ($8.13 → $0.45)
|
||||
|
||||
---
|
||||
|
||||
## Common Issues
|
||||
|
||||
### Issue: "Failed to connect to session"
|
||||
|
||||
**Cause:** Session closed between `.sessions()` and `.connect()` calls
|
||||
|
||||
**Solution:**
|
||||
```typescript
|
||||
const freeSession = sessions.find(s => !s.connectionId);
|
||||
if (freeSession) {
|
||||
try {
|
||||
return await puppeteer.connect(env.MYBROWSER, freeSession.sessionId);
|
||||
} catch (error) {
|
||||
console.log("Session closed, launching new browser");
|
||||
return await puppeteer.launch(env.MYBROWSER);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Issue: Sessions timing out too quickly
|
||||
|
||||
**Cause:** Default 60s idle timeout
|
||||
|
||||
**Solution:** Extend with keep_alive:
|
||||
```typescript
|
||||
const browser = await puppeteer.launch(env.MYBROWSER, {
|
||||
keep_alive: 300000 // 5 minutes
|
||||
});
|
||||
```
|
||||
|
||||
### Issue: Rate limit reached
|
||||
|
||||
**Cause:** Too many concurrent browsers or launches per minute
|
||||
|
||||
**Solution:** Check limits before launching:
|
||||
```typescript
|
||||
const limits = await puppeteer.limits(env.MYBROWSER);
|
||||
if (limits.allowedBrowserAcquisitions === 0) {
|
||||
return new Response("Rate limit reached", { status: 429 });
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Reference
|
||||
|
||||
- **Official Docs**: https://developers.cloudflare.com/browser-rendering/workers-bindings/reuse-sessions/
|
||||
- **Limits**: https://developers.cloudflare.com/browser-rendering/platform/limits/
|
||||
- **Pricing**: https://developers.cloudflare.com/browser-rendering/platform/pricing/
|
||||
|
||||
---
|
||||
|
||||
**Last Updated**: 2025-10-22
|
||||
Reference in New Issue
Block a user