Initial commit

2025-11-30 08:43:37 +08:00
commit 22c15fd73f
12 changed files with 1478 additions and 0 deletions
--- a/commands/steel-create.md
+++ b/commands/steel-create.md
@@ -0,0 +1,106 @@
+---
+description: Create a new Steel browser automation project with best practices
+---
+
+# Steel Project Creator
+
+Help the user create a new Steel automation project. Guide them through setup and generate starter code.
+
+## Steps
+
+**Note**: If the user has Steel CLI installed, you can use `steel forge <template>` to create projects instantly from official templates. Check with `which steel` or suggest installing: `npm install -g @steel-dev/cli`
+
+Available templates: `playwright`, `playwright-py`, `puppeteer`, `browser-use`, `oai-cua`, and more.
+
+1. **Ask about the project**:
+   - Project name
+   - Use case (web scraping, testing, e-commerce automation, etc.)
+   - Language preference (TypeScript or Python)
+
+2. **Create project structure** (or use `steel forge` if CLI available):
+   ```
+   project-name/
+   ├── package.json (or requirements.txt)
+   ├── .env.example
+   ├── src/
+   │   └── index.ts (or main.py)
+   └── README.md
+   ```
+
+3. **Generate starter code** based on use case:
+
+   **TypeScript Example**:
+   ```typescript
+   import { Steel } from 'steel-sdk';
+   
+   const client = new Steel({
+     steelAPIKey: process.env.STEEL_API_KEY!
+   });
+   
+   async function main() {
+     const session = await client.sessions.create({
+       dimensions: { width: 1280, height: 800 }
+     });
+     
+     console.log('Session created:', session.id);
+     console.log('Live view:', session.sessionViewerUrl);
+     
+     // Your automation code here
+     
+     await client.sessions.release(session.id);
+   }
+   
+   main().catch(console.error);
+   ```
+
+   **Python Example**:
+   ```python
+   from steel import Steel
+   import os
+   
+   client = Steel(steel_api_key=os.getenv('STEEL_API_KEY'))
+   
+   def main():
+       session = client.sessions.create(
+           dimensions={'width': 1280, 'height': 800}
+       )
+       
+       print(f'Session created: {session.id}')
+       print(f'Live view: {session.session_viewer_url}')
+       
+       # Your automation code here
+       
+       client.sessions.release(session.id)
+   
+   if __name__ == '__main__':
+       main()
+   ```
+
+4. **Create .env.example**:
+   ```
+   STEEL_API_KEY=your_api_key_here
+   ```
+
+5. **Create README.md** with:
+   - Project description
+   - Setup instructions
+   - How to run
+   - Link to Steel docs: https://docs.steel.dev
+
+## Key Steel Features to Mention
+
+- **Live Sessions**: Use `session.sessionViewerUrl` to watch automation in real-time
+- **Proxy Support**: Add `useProxy: true` for geo-specific automation
+- **CAPTCHA Solving**: Add `solveCaptcha: true` to handle CAPTCHAs
+- **File Operations**: Use `/v1/sessions/:id/files` API for file uploads/downloads
+- **Multi-tab**: Steel supports multiple tabs per session
+
+## Best Practices
+
+- Always release sessions when done
+- Use environment variables for API keys
+- Enable session viewer URLs during development
+- Handle errors gracefully
+- Set appropriate timeouts
+
+After creating the project, guide the user through installing dependencies and running the first session.
--- a/commands/steel-debug.md
+++ b/commands/steel-debug.md
@@ -0,0 +1,191 @@
+---
+description: Debug Steel automation issues and provide solutions
+---
+
+# Steel Debugging Assistant
+
+Help the user debug Steel automation issues using live sessions and error analysis.
+
+## Debugging Approach
+
+1. **Ask to see the error** - Get the full error message and stack trace
+2. **Check the live session** - Ask if they're using `sessionViewerUrl` to see what's happening
+3. **Identify the issue** - Match error patterns to common Steel issues
+4. **Provide specific fix** - Give working code that solves the problem
+
+## Steel CLI for Debugging
+
+If user has Steel CLI (`@steel-dev/cli`), suggest:
+- `steel config` - Verify API key and configuration
+- `steel run <template> --view` - Run working example to compare
+- `steel browser start --verbose` - Start local browser with detailed logs
+
+Install if needed: `npm install -g @steel-dev/cli`
+
+## Common Steel Issues & Fixes
+
+### Issue: Session Creation Timeout
+**Symptoms**: `Session creation timed out` or similar errors
+
+**Solution**:
+```typescript
+const session = await client.sessions.create({
+  sessionTimeout: 60000, // Increase timeout to 60s
+});
+```
+
+### Issue: Selector Not Found
+**Symptoms**: `TimeoutError: waiting for selector failed`
+
+**Solution**:
+```typescript
+// Wait for page to fully load
+await page.waitForLoadState('networkidle');
+
+// Try multiple selectors
+const selectors = [
+  '[data-testid="submit"]',
+  '#submit-button',
+  'button:has-text("Submit")'
+];
+
+for (const selector of selectors) {
+  try {
+    await page.waitForSelector(selector, { timeout: 5000 });
+    break;
+  } catch (e) {
+    continue;
+  }
+}
+```
+
+### Issue: WebSocket Connection Failed
+**Symptoms**: `WebSocket connection failed` or `CDP connection error`
+
+**Check**:
+```typescript
+// Ensure API key is passed correctly
+const wsUrl = `${session.websocketUrl}?apiKey=${process.env.STEEL_API_KEY}`;
+const browser = await chromium.connectOverCDP(wsUrl);
+```
+
+### Issue: Session Not Releasing
+**Symptoms**: Running out of concurrent sessions
+
+**Solution**:
+```typescript
+// Always use try-finally
+const session = await client.sessions.create();
+try {
+  // Your automation code
+} finally {
+  await client.sessions.release(session.id);
+}
+```
+
+### Issue: Page Loading Too Slowly
+**Symptoms**: Timeouts during page navigation
+
+**Solution**:
+```typescript
+const session = await client.sessions.create({
+  blockAds: true, // Block ads for faster loading
+  dimensions: { width: 1280, height: 800 }
+});
+
+await page.goto(url, { 
+  waitUntil: 'domcontentloaded' // Don't wait for everything
+});
+```
+
+### Issue: CAPTCHA Blocking Automation
+**Symptoms**: Automation stops at CAPTCHA pages
+
+**Solution**:
+```typescript
+const session = await client.sessions.create({
+  solveCaptcha: true, // Enable CAPTCHA solving
+  useProxy: true      // Use residential proxies
+});
+```
+
+### Issue: Cannot Find Elements in Iframe
+**Symptoms**: Elements exist but selector fails
+
+**Solution**:
+```typescript
+// Wait for iframe
+const frameElement = await page.waitForSelector('iframe');
+const frame = await frameElement.contentFrame();
+
+// Search inside iframe
+await frame.waitForSelector('[data-testid="target"]');
+```
+
+### Issue: Dynamic Content Not Loading
+**Symptoms**: Elements appear later but selector times out
+
+**Solution**:
+```typescript
+// Wait for network to be idle
+await page.waitForLoadState('networkidle');
+
+// Or wait for specific element
+await page.waitForSelector('[data-testid="loaded-content"]', {
+  state: 'visible',
+  timeout: 15000
+});
+```
+
+## Debugging Workflow
+
+When user reports an issue:
+
+1. **Get context**:
+   - What are they trying to do?
+   - What error message do they see?
+   - Can they share the code?
+
+2. **Use live session**:
+   ```typescript
+   const session = await client.sessions.create();
+   console.log('Debug at:', session.sessionViewerUrl);
+   // User can watch what's happening in real-time
+   ```
+
+3. **Add debugging output**:
+   ```typescript
+   console.log('Step 1: Creating session');
+   const session = await client.sessions.create();
+   console.log('Session ID:', session.id);
+   
+   console.log('Step 2: Connecting browser');
+   const browser = await chromium.connectOverCDP(/*...*/);
+   
+   console.log('Step 3: Navigating to page');
+   await page.goto(url);
+   console.log('Current URL:', page.url());
+   ```
+
+4. **Test selectors**:
+   ```typescript
+   // Check if element exists
+   const element = await page.$('[data-testid="target"]');
+   if (element) {
+     console.log('Element found!');
+   } else {
+     console.log('Element not found, trying alternatives...');
+   }
+   ```
+
+5. **Provide working solution** with explanation
+
+## When to Check Steel Status
+
+If issues persist, suggest checking:
+- Steel Cloud status: https://status.steel.dev
+- API key validity: Check in Steel dashboard
+- Session limits: Check concurrent session limits
+- Network issues: Test from different network
+
+Always focus on practical, immediate solutions over theoretical debugging approaches.
--- a/commands/steel-deploy.md
+++ b/commands/steel-deploy.md
@@ -0,0 +1,271 @@
+---
+description: Help deploy Steel automation to production environments
+---
+
+# Steel Deployment Assistant
+
+Help the user deploy Steel automation to production with best practices and configurations.
+
+## Deployment Options
+
+### 1. Docker Deployment
+
+**Create Dockerfile**:
+```dockerfile
+FROM node:18-alpine
+
+WORKDIR /app
+
+# Install dependencies
+COPY package*.json ./
+RUN npm ci --only=production
+
+# Copy source
+COPY . .
+
+# Build if using TypeScript
+RUN npm run build
+
+# Run the automation
+CMD ["node", "dist/index.js"]
+```
+
+**Docker Compose with Steel**:
+```yaml
+version: '3.8'
+services:
+  steel-automation:
+    build: .
+    environment:
+      - STEEL_API_KEY=${STEEL_API_KEY}
+      - NODE_ENV=production
+    restart: unless-stopped
+```
+
+### 2. Serverless Deployment (AWS Lambda, etc.)
+
+**Key Considerations**:
+- Steel handles the browser, your function just needs to call the API
+- Set appropriate timeouts (Lambda: 15 min max)
+- Use environment variables for API keys
+
+**Lambda Handler Example**:
+```typescript
+import { Steel } from 'steel-sdk';
+
+export const handler = async (event: any) => {
+  const client = new Steel({ 
+    steelAPIKey: process.env.STEEL_API_KEY! 
+  });
+  
+  const session = await client.sessions.create();
+  
+  try {
+    // Your automation logic
+    const result = await runAutomation(session);
+    return { statusCode: 200, body: JSON.stringify(result) };
+  } finally {
+    await client.sessions.release(session.id);
+  }
+};
+```
+
+### 3. Scheduled Jobs (Cron, GitHub Actions)
+
+**GitHub Actions Example**:
+```yaml
+name: Daily Steel Scraping
+
+on:
+  schedule:
+    - cron: '0 9 * * *'  # Daily at 9 AM UTC
+  workflow_dispatch:
+
+jobs:
+  scrape:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      
+      - name: Setup Node.js
+        uses: actions/setup-node@v3
+        with:
+          node-version: '18'
+      
+      - name: Install dependencies
+        run: npm ci
+      
+      - name: Run Steel automation
+        env:
+          STEEL_API_KEY: ${{ secrets.STEEL_API_KEY }}
+        run: npm start
+```
+
+### 4. Kubernetes Deployment
+
+**Deployment YAML**:
+```yaml
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: steel-automation
+spec:
+  replicas: 3
+  template:
+    spec:
+      containers:
+      - name: automation
+        image: your-registry/steel-automation:latest
+        env:
+        - name: STEEL_API_KEY
+          valueFrom:
+            secretKeyRef:
+              name: steel-secrets
+              key: api-key
+        resources:
+          limits:
+            memory: "512Mi"
+            cpu: "500m"
+```
+
+## Production Best Practices
+
+### Environment Configuration
+```typescript
+const config = {
+  steel: {
+    apiKey: process.env.STEEL_API_KEY,
+    sessionTimeout: parseInt(process.env.STEEL_TIMEOUT || '60000'),
+    retries: parseInt(process.env.STEEL_RETRIES || '3')
+  },
+  // Other config...
+};
+```
+
+### Error Handling & Retries
+```typescript
+async function runWithRetry(fn: () => Promise<any>, retries = 3) {
+  for (let i = 0; i < retries; i++) {
+    try {
+      return await fn();
+    } catch (error) {
+      console.error(`Attempt ${i + 1} failed:`, error);
+      if (i === retries - 1) throw error;
+      await new Promise(r => setTimeout(r, 1000 * (i + 1))); // Backoff
+    }
+  }
+}
+```
+
+### Session Management
+```typescript
+// Always clean up sessions
+const session = await client.sessions.create();
+try {
+  await runAutomation(session);
+} catch (error) {
+  console.error('Automation failed:', error);
+  console.error('Session viewer:', session.sessionViewerUrl);
+  throw error;
+} finally {
+  await client.sessions.release(session.id);
+}
+```
+
+### Logging & Monitoring
+```typescript
+// Structured logging
+console.log(JSON.stringify({
+  timestamp: new Date().toISOString(),
+  level: 'info',
+  message: 'Session created',
+  sessionId: session.id,
+  viewerUrl: session.sessionViewerUrl
+}));
+
+// Track metrics
+const startTime = Date.now();
+await runAutomation(session);
+const duration = Date.now() - startTime;
+console.log(JSON.stringify({
+  metric: 'automation_duration_ms',
+  value: duration
+}));
+```
+
+### Security Checklist
+- ✅ Store API keys in environment variables or secrets management
+- ✅ Never commit `.env` files with real keys
+- ✅ Use least-privilege API keys when possible
+- ✅ Enable rate limiting in your application
+- ✅ Monitor for unusual usage patterns
+- ✅ Set session timeouts to prevent runaway sessions
+
+### Cost Optimization
+```typescript
+// Reuse sessions when possible
+const sessionPool = new SessionPool(client, { maxSize: 5 });
+
+// Use appropriate dimensions
+const session = await client.sessions.create({
+  dimensions: { width: 1280, height: 800 } // Smaller = faster
+});
+
+// Block unnecessary resources
+const session = await client.sessions.create({
+  blockAds: true // Faster and cheaper
+});
+```
+
+## Deployment Checklist
+
+Before deploying, verify:
+
+1. **Environment Variables Set**:
+   - `STEEL_API_KEY` configured
+   - Other necessary config variables
+
+2. **Error Handling**:
+   - Try-catch blocks around Steel operations
+   - Proper session cleanup in finally blocks
+   - Retry logic for transient failures
+
+3. **Resource Limits**:
+   - Appropriate timeouts configured
+   - Memory limits set (if containerized)
+   - Concurrent session limits considered
+
+4. **Monitoring**:
+   - Logging configured
+   - Error tracking (Sentry, etc.) set up
+   - Metrics collection enabled
+
+5. **Testing**:
+   - Tested in staging environment
+   - Load tested if expecting high volume
+   - Verified session cleanup works
+
+## Common Deployment Issues
+
+### Issue: API Key Not Found
+```typescript
+if (!process.env.STEEL_API_KEY) {
+  throw new Error('STEEL_API_KEY environment variable not set');
+}
+```
+
+### Issue: Timeouts in Production
+```typescript
+const session = await client.sessions.create({
+  sessionTimeout: 120000 // 2 minutes for production
+});
+```
+
+### Issue: Too Many Concurrent Sessions
+```typescript
+// Implement session pooling or queuing
+const queue = new PQueue({ concurrency: 5 });
+await queue.add(() => runAutomation());
+```
+
+Guide users through their specific deployment scenario with practical, working configurations.
--- a/commands/steel-dev.md
+++ b/commands/steel-dev.md
@@ -0,0 +1,93 @@
+---
+description: Help develop and debug Steel automation with live session monitoring
+---
+
+# Steel Development Assistant
+
+Help the user develop Steel automation with live debugging capabilities and best practices.
+
+## What You Do
+
+1. **Enable Live Debugging**: Always suggest using `sessionViewerUrl` during development
+2. **Provide Steel-specific Code**: Generate code using Steel SDK patterns
+3. **Handle Common Scenarios**: Help with selectors, navigation, data extraction
+4. **Suggest Optimizations**: Recommend session reuse, proper timeouts, error handling
+
+## Steel CLI Tips
+
+If user has Steel CLI (`@steel-dev/cli`), you can suggest:
+- `steel run <template> --view` - Run cookbook examples with live session viewer
+- `steel browser start` - Start local Steel browser for development
+- `steel config` - View current Steel configuration
+
+Install if needed: `npm install -g @steel-dev/cli`
+
+## Common Development Tasks
+
+### Connect to Live Session
+```typescript
+const session = await client.sessions.create({
+  dimensions: { width: 1280, height: 800 }
+});
+
+console.log('Watch live:', session.sessionViewerUrl);
+```
+
+### Use Playwright with Steel
+```typescript
+import { chromium } from 'playwright';
+
+const browser = await chromium.connectOverCDP(
+  `${session.websocketUrl}?apiKey=${process.env.STEEL_API_KEY}`
+);
+
+const context = browser.contexts[0];
+const page = context.pages[0];
+
+await page.goto('https://example.com');
+```
+
+### Handle Common Issues
+
+**Selector Not Found**:
+```typescript
+// Use multiple wait strategies
+await page.waitForLoadState('networkidle');
+await page.waitForSelector('[data-testid="element"]', { 
+  timeout: 10000 
+});
+```
+
+**Session Timeout**:
+```typescript
+const session = await client.sessions.create({
+  sessionTimeout: 300000 // 5 minutes
+});
+```
+
+**Proxy Issues**:
+```typescript
+const session = await client.sessions.create({
+  useProxy: {
+    country: 'US'
+  }
+});
+```
+
+## Steel-Specific Tips
+
+- **Fast Session Creation**: Steel creates sessions in ~400ms
+- **Multi-tab Support**: Use Playwright's `context.newPage()` for multiple tabs
+- **Live Embedding**: Embed `sessionViewerUrl` in your app for human-in-the-loop
+- **File Operations**: Use Steel's file API for uploads/downloads
+- **Ad Blocking**: `blockAds: true` by default for faster loading
+
+## When User Asks for Help
+
+1. Ask to see their current code
+2. Check if they're using Steel SDK correctly
+3. Suggest using `sessionViewerUrl` to see what's happening
+4. Provide specific code fixes with Steel patterns
+5. Link to relevant Steel docs when needed
+
+Always prioritize practical, working code over complex abstractions.
--- a/commands/steel-test.md
+++ b/commands/steel-test.md
@@ -0,0 +1,164 @@
+---
+description: Help write and run tests for Steel automation projects
+---
+
+# Steel Testing Assistant
+
+Help the user write effective tests for Steel automation using standard testing frameworks.
+
+## Test Approaches
+
+### Unit Tests
+Test individual functions and utilities:
+```typescript
+// tests/utils.test.ts
+import { generateSelector } from '../src/utils';
+
+describe('generateSelector', () => {
+  it('should create optimal selector', () => {
+    const element = { id: 'submit', class: 'btn' };
+    expect(generateSelector(element)).toBe('#submit');
+  });
+});
+```
+
+### Integration Tests
+Test Steel session management:
+```typescript
+// tests/session.test.ts
+import { Steel } from 'steel-sdk';
+
+describe('Steel Session', () => {
+  let client: Steel;
+  
+  beforeAll(() => {
+    client = new Steel({ steelAPIKey: process.env.STEEL_API_KEY! });
+  });
+
+  it('should create session', async () => {
+    const session = await client.sessions.create();
+    expect(session.id).toBeDefined();
+    await client.sessions.release(session.id);
+  });
+
+  it('should connect with Playwright', async () => {
+    const session = await client.sessions.create();
+    const browser = await chromium.connectOverCDP(
+      `${session.websocketUrl}?apiKey=${process.env.STEEL_API_KEY}`
+    );
+    
+    expect(browser.contexts().length).toBeGreaterThan(0);
+    
+    await browser.close();
+    await client.sessions.release(session.id);
+  });
+});
+```
+
+### E2E Tests
+Test complete automation workflows:
+```typescript
+// tests/e2e/scraping.test.ts
+describe('Product Scraping', () => {
+  it('should scrape product data', async () => {
+    const client = new Steel({ steelAPIKey: process.env.STEEL_API_KEY! });
+    const session = await client.sessions.create();
+    
+    const browser = await chromium.connectOverCDP(
+      `${session.websocketUrl}?apiKey=${process.env.STEEL_API_KEY}`
+    );
+    const page = browser.contexts()[0].pages()[0];
+    
+    await page.goto('https://example.com/products');
+    const title = await page.locator('h1').first().textContent();
+    
+    expect(title).toBeTruthy();
+    
+    await browser.close();
+    await client.sessions.release(session.id);
+  });
+});
+```
+
+## Test Best Practices
+
+1. **Always Clean Up Sessions**:
+   ```typescript
+   afterEach(async () => {
+     if (session) {
+       await client.sessions.release(session.id);
+     }
+   });
+   ```
+
+2. **Use Longer Timeouts**: Steel automation can take time
+   ```typescript
+   jest.setTimeout(30000); // 30 seconds
+   ```
+
+3. **Handle Errors Gracefully**:
+   ```typescript
+   try {
+     await page.goto('https://example.com');
+   } catch (error) {
+     console.log('Session viewer:', session.sessionViewerUrl);
+     throw error;
+   }
+   ```
+
+4. **Test with Live Sessions**: Use `sessionViewerUrl` to debug test failures
+
+5. **Mock External Services**: Don't test third-party APIs, focus on your Steel code
+
+## Common Test Patterns
+
+### Session Creation Test
+```typescript
+it('should create session within timeout', async () => {
+  const start = Date.now();
+  const session = await client.sessions.create();
+  const duration = Date.now() - start;
+  
+  expect(duration).toBeLessThan(5000); // Should be fast
+  await client.sessions.release(session.id);
+});
+```
+
+### Selector Reliability Test
+```typescript
+it('should find element with selector', async () => {
+  const session = await client.sessions.create();
+  const browser = await chromium.connectOverCDP(/*...*/);
+  const page = browser.contexts()[0].pages()[0];
+  
+  await page.goto('https://example.com');
+  const element = await page.waitForSelector('[data-testid="target"]');
+  
+  expect(element).toBeTruthy();
+});
+```
+
+### Data Extraction Test
+```typescript
+it('should extract correct data', async () => {
+  // Setup session and page
+  const data = await page.evaluate(() => {
+    return {
+      title: document.querySelector('h1')?.textContent,
+      price: document.querySelector('.price')?.textContent
+    };
+  });
+  
+  expect(data.title).toBeTruthy();
+  expect(data.price).toMatch(/\$\d+/);
+});
+```
+
+## Running Tests
+
+Guide users to use standard test runners:
+- Jest: `npm test` or `jest`
+- Vitest: `vitest run`
+- Pytest: `pytest tests/`
+
+Always provide working test examples they can copy and modify.