Initial commit

This commit is contained in:
Zhongwei Li
2025-11-30 08:24:46 +08:00
commit 49178918d7
24 changed files with 3940 additions and 0 deletions

413
references/api-reference.md Normal file
View File

@@ -0,0 +1,413 @@
# ElevenLabs Agents API Reference
## Base URL
```
https://api.elevenlabs.io/v1/convai
```
## Authentication
All requests require an API key in the header:
```bash
curl -H "xi-api-key: YOUR_API_KEY" https://api.elevenlabs.io/v1/convai/agents
```
---
## Agents
### Create Agent
**Endpoint**: `POST /agents/create`
**Request Body**:
```json
{
"name": "Support Agent",
"conversation_config": {
"agent": {
"prompt": {
"prompt": "You are a helpful support agent.",
"llm": "gpt-4o",
"temperature": 0.7,
"max_tokens": 500,
"tool_ids": ["tool_123"],
"knowledge_base": ["doc_456"]
},
"first_message": "Hello! How can I help?",
"language": "en"
},
"tts": {
"model_id": "eleven_turbo_v2_5",
"voice_id": "voice_abc123",
"stability": 0.5,
"similarity_boost": 0.75,
"speed": 1.0
},
"asr": {
"quality": "high",
"provider": "deepgram"
},
"turn": {
"mode": "normal"
}
}
}
```
**Response**:
```json
{
"agent_id": "agent_abc123",
"name": "Support Agent",
"created_at": "2025-11-03T12:00:00Z"
}
```
### Update Agent
**Endpoint**: `PATCH /agents/:agent_id`
**Request Body**: Same as Create Agent
### Get Agent
**Endpoint**: `GET /agents/:agent_id`
**Response**:
```json
{
"agent_id": "agent_abc123",
"name": "Support Agent",
"conversation_config": { ... },
"created_at": "2025-11-03T12:00:00Z",
"updated_at": "2025-11-03T14:00:00Z"
}
```
### List Agents
**Endpoint**: `GET /agents`
**Response**:
```json
{
"agents": [
{
"agent_id": "agent_abc123",
"name": "Support Agent",
"created_at": "2025-11-03T12:00:00Z"
}
]
}
```
### Delete Agent
**Endpoint**: `DELETE /agents/:agent_id`
**Response**:
```json
{
"success": true
}
```
---
## Conversations
### Create Conversation
**Endpoint**: `POST /conversations/create`
**Request Body**:
```json
{
"agent_id": "agent_abc123",
"dynamic_variables": {
"user_name": "John",
"account_tier": "premium"
},
"overrides": {
"agent": {
"prompt": {
"prompt": "Custom prompt override"
}
}
}
}
```
**Response**:
```json
{
"conversation_id": "conv_xyz789",
"signed_url": "wss://api.elevenlabs.io/v1/convai/...",
"created_at": "2025-11-03T12:00:00Z"
}
```
### Get Conversation
**Endpoint**: `GET /conversations/:conversation_id`
**Response**:
```json
{
"conversation_id": "conv_xyz789",
"agent_id": "agent_abc123",
"transcript": "...",
"duration_seconds": 120,
"status": "completed",
"created_at": "2025-11-03T12:00:00Z",
"ended_at": "2025-11-03T12:02:00Z"
}
```
---
## Knowledge Base
### Upload Document
**Endpoint**: `POST /knowledge-base/upload`
**Request Body** (multipart/form-data):
```
file: <binary>
name: "Support Documentation"
```
**Response**:
```json
{
"document_id": "doc_456",
"name": "Support Documentation",
"status": "processing"
}
```
### Compute RAG Index
**Endpoint**: `POST /knowledge-base/:document_id/rag-index`
**Request Body**:
```json
{
"embedding_model": "e5_mistral_7b"
}
```
**Response**:
```json
{
"document_id": "doc_456",
"status": "computing"
}
```
### Get RAG Index Status
**Endpoint**: `GET /knowledge-base/:document_id/rag-index`
**Response**:
```json
{
"document_id": "doc_456",
"status": "ready",
"embedding_model": "e5_mistral_7b",
"created_at": "2025-11-03T12:00:00Z"
}
```
---
## Tools
### Create Webhook Tool
**Endpoint**: `POST /tools/webhook`
**Request Body**:
```json
{
"name": "get_weather",
"description": "Fetch current weather for a city",
"url": "https://api.weather.com/v1/current",
"method": "GET",
"parameters": {
"type": "object",
"properties": {
"city": {
"type": "string",
"description": "City name"
}
},
"required": ["city"]
},
"headers": {
"Authorization": "Bearer {{secret__weather_api_key}}"
}
}
```
**Response**:
```json
{
"tool_id": "tool_123",
"name": "get_weather",
"created_at": "2025-11-03T12:00:00Z"
}
```
---
## Testing
### Simulate Conversation
**Endpoint**: `POST /agents/:agent_id/simulate`
**Request Body**:
```json
{
"scenario": "Customer requests refund",
"user_messages": [
"I want a refund for order #12345",
"I ordered it last week"
],
"success_criteria": [
"Agent acknowledges request",
"Agent provides timeline"
]
}
```
**Response**:
```json
{
"simulation_id": "sim_123",
"passed": true,
"transcript": "...",
"evaluation": {
"criteria_met": 2,
"criteria_total": 2,
"details": [
{
"criterion": "Agent acknowledges request",
"passed": true
},
{
"criterion": "Agent provides timeline",
"passed": true
}
]
}
}
```
---
## Error Codes
| Code | Meaning | Solution |
|------|---------|----------|
| 400 | Bad Request | Check request body format |
| 401 | Unauthorized | Verify API key is correct |
| 403 | Forbidden | Check agent visibility settings |
| 404 | Not Found | Verify resource ID exists |
| 429 | Rate Limited | Implement backoff strategy |
| 500 | Server Error | Retry with exponential backoff |
---
## Rate Limits
- **Standard Tier**: 100 requests/minute
- **Pro Tier**: 500 requests/minute
- **Enterprise Tier**: Custom limits
**Rate Limit Headers**:
```
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 95
X-RateLimit-Reset: 1730640000
```
---
## Pagination
**Query Parameters**:
```
?page=1&per_page=50
```
**Response Headers**:
```
X-Total-Count: 250
X-Page: 1
X-Per-Page: 50
```
---
## Webhook Events
### Post-Call Webhook
**Event Type**: `post_call_transcription`
**Payload**:
```json
{
"type": "post_call_transcription",
"data": {
"conversation_id": "conv_xyz789",
"agent_id": "agent_abc123",
"transcript": "...",
"duration_seconds": 120,
"analysis": {
"sentiment": "positive",
"resolution": true,
"extracted_data": {}
}
},
"event_timestamp": "2025-11-03T12:02:00Z"
}
```
**Verification** (HMAC SHA-256):
```typescript
import crypto from 'crypto';
const signature = request.headers['elevenlabs-signature'];
const payload = JSON.stringify(request.body);
const hmac = crypto
.createHmac('sha256', process.env.WEBHOOK_SECRET)
.update(payload)
.digest('hex');
if (signature !== hmac) {
// Invalid signature
}
```
---
## SDK vs API
| Feature | SDK | API |
|---------|-----|-----|
| WebSocket Connection | ✅ | ❌ |
| Client Tools | ✅ | ❌ |
| Real-time Events | ✅ | ❌ |
| Agent Management | ❌ | ✅ |
| Tool Management | ❌ | ✅ |
| Knowledge Base | ❌ | ✅ |
**Recommendation**: Use SDK for conversations, API for agent management.

274
references/cli-commands.md Normal file
View File

@@ -0,0 +1,274 @@
# CLI Commands Reference
## Installation
```bash
npm install -g @elevenlabs/cli
# or
pnpm install -g @elevenlabs/cli
```
---
## Authentication
### Login
```bash
elevenlabs auth login
```
### Check Current User
```bash
elevenlabs auth whoami
```
### Set Residency
```bash
elevenlabs auth residency eu-residency
# Options: global | eu-residency | in-residency
```
### Logout
```bash
elevenlabs auth logout
```
---
## Project Initialization
### Initialize New Project
```bash
elevenlabs agents init
```
### Recreate Project Structure
```bash
elevenlabs agents init --override
```
---
## Agent Management
### Add Agent
```bash
elevenlabs agents add "Agent Name" --template TEMPLATE
```
**Templates**: default | minimal | voice-only | text-only | customer-service | assistant
### Push to Platform
```bash
# Push all agents
elevenlabs agents push
# Push specific agent
elevenlabs agents push --agent "Agent Name"
# Push to environment
elevenlabs agents push --env prod
# Dry run (preview changes)
elevenlabs agents push --dry-run
```
### Pull from Platform
```bash
# Pull all agents
elevenlabs agents pull
# Pull specific agent
elevenlabs agents pull --agent "Agent Name"
```
### List Agents
```bash
elevenlabs agents list
```
### Check Sync Status
```bash
elevenlabs agents status
```
### Delete Agent
```bash
elevenlabs agents delete AGENT_ID
```
### Generate Widget
```bash
elevenlabs agents widget "Agent Name"
```
---
## Tool Management
### Add Webhook Tool
```bash
elevenlabs tools add-webhook "Tool Name" --config-path tool_configs/tool.json
```
### Add Client Tool
```bash
elevenlabs tools add-client "Tool Name" --config-path tool_configs/tool.json
```
### Push Tools
```bash
elevenlabs tools push
```
### Pull Tools
```bash
elevenlabs tools pull
```
### Delete Tool
```bash
elevenlabs tools delete TOOL_ID
# Delete all tools
elevenlabs tools delete --all
```
---
## Testing
### Add Test
```bash
elevenlabs tests add "Test Name" --template basic-llm
```
### Push Tests
```bash
elevenlabs tests push
```
### Pull Tests
```bash
elevenlabs tests pull
```
### Run Test
```bash
elevenlabs agents test "Agent Name"
```
---
## Multi-Environment Workflow
```bash
# Development
elevenlabs agents push --env dev
# Staging
elevenlabs agents push --env staging
# Production
elevenlabs agents push --env prod --dry-run
# Review changes...
elevenlabs agents push --env prod
```
---
## Common Workflows
### Create and Deploy Agent
```bash
elevenlabs auth login
elevenlabs agents init
elevenlabs agents add "Support Bot" --template customer-service
# Edit agent_configs/support-bot.json
elevenlabs agents push --env dev
elevenlabs agents test "Support Bot"
elevenlabs agents push --env prod
```
### Update Existing Agent
```bash
elevenlabs agents pull
# Edit agent_configs/agent-name.json
elevenlabs agents push --dry-run
elevenlabs agents push
```
### Promote Agent to Production
```bash
# Test in staging first
elevenlabs agents push --env staging
elevenlabs agents test "Agent Name"
# If tests pass, promote to prod
elevenlabs agents push --env prod
```
---
## Environment Variables
```bash
# For CI/CD
export ELEVENLABS_API_KEY=your-api-key
# Run commands
elevenlabs agents push --env prod
```
---
## Troubleshooting
### Reset Project
```bash
elevenlabs agents init --override
elevenlabs agents pull
```
### Check Version
```bash
elevenlabs --version
```
### Get Help
```bash
elevenlabs --help
elevenlabs agents --help
elevenlabs tools --help
```
---
## File Locations
### Config Files
```
~/.elevenlabs/api_key # API key (if not using keychain)
```
### Project Files
```
./agents.json # Agent registry
./tools.json # Tool registry
./tests.json # Test registry
./agent_configs/*.json # Individual agent configs
./tool_configs/*.json # Individual tool configs
./test_configs/*.json # Individual test configs
```
---
## Best Practices
1. **Always use --dry-run** before pushing to production
2. **Commit configs to Git** for version control
3. **Use environment-specific configs** (dev/staging/prod)
4. **Test agents** before deploying
5. **Pull before editing** to avoid conflicts
6. **Use templates** for consistency
7. **Document changes** in commit messages

View File

@@ -0,0 +1,227 @@
# Privacy & Compliance Guide
## GDPR Compliance
### Data Retention
**Default**: 2 years (730 days)
```json
{
"privacy": {
"transcripts": {
"retention_days": 730
},
"audio": {
"retention_days": 730
}
}
}
```
### Right to Be Forgotten
Enable data deletion requests:
```typescript
await client.conversations.delete(conversation_id);
```
### Data Residency
```typescript
const { startConversation } = useConversation({
serverLocation: 'eu-residency' // GDPR-compliant EU data centers
});
```
### User Consent
Inform users before recording:
```json
{
"first_message": "This call will be recorded for quality and training purposes. Do you consent?"
}
```
---
## HIPAA Compliance
### Data Retention
**Minimum**: 6 years (2190 days)
```json
{
"privacy": {
"transcripts": {
"retention_days": 2190
},
"audio": {
"retention_days": 2190
}
}
}
```
### Encryption
- **In Transit**: TLS 1.3 (automatic)
- **At Rest**: AES-256 (automatic)
### Business Associate Agreement (BAA)
Contact ElevenLabs for HIPAA BAA.
### PHI Handling
**Never**:
- Store PHI in dynamic variables
- Log PHI in tool parameters
- Send PHI to third-party tools without BAA
**Always**:
- Use secure authentication
- Verify patient identity
- Document access logs
---
## SOC 2 Compliance
### Security Controls
✅ Encryption in transit and at rest (automatic)
✅ Access controls (API key management)
✅ Audit logs (conversation history)
✅ Incident response (automatic backups)
### Best Practices
```json
{
"authentication": {
"type": "signed_url", // Most secure
"session_duration": 3600 // 1 hour max
}
}
```
---
## Regional Compliance
### US Residency
```typescript
serverLocation: 'us'
```
### EU Residency (GDPR)
```typescript
serverLocation: 'eu-residency'
```
### India Residency
```typescript
serverLocation: 'in-residency'
```
---
## Zero Retention Mode
**Maximum Privacy**: Immediately delete all data after conversation ends.
```json
{
"privacy": {
"zero_retention": true
}
}
```
**Limitations**:
- No conversation history
- No analytics
- No post-call webhooks
- No MCP tool integrations
---
## PCI DSS (Payment Card Industry)
### Never:
❌ Store credit card numbers in conversation logs
❌ Send credit card data to LLM
❌ Log CVV or PIN numbers
### Always:
✅ Use PCI-compliant payment processors (Stripe, PayPal)
✅ Tokenize payment data
✅ Use DTMF keypad for card entry (telephony)
### Example: Secure Payment Collection
```json
{
"system_tools": [
{
"name": "dtmf_playpad",
"description": "Display keypad for secure card entry"
}
]
}
```
---
## Compliance Checklist
### GDPR
- [ ] Data retention ≤ 2 years (or justify longer)
- [ ] EU data residency enabled
- [ ] User consent obtained before recording
- [ ] Data deletion process implemented
- [ ] Privacy policy updated
### HIPAA
- [ ] Data retention ≥ 6 years
- [ ] BAA signed with ElevenLabs
- [ ] Encryption enabled (automatic)
- [ ] Access logs maintained
- [ ] Staff trained on PHI handling
### SOC 2
- [ ] API key security (never expose in client)
- [ ] Use signed URLs for authentication
- [ ] Monitor access logs
- [ ] Incident response plan documented
### PCI DSS
- [ ] Never log card data
- [ ] Use tokenization for payments
- [ ] DTMF keypad for card entry
- [ ] PCI-compliant payment processor
---
## Monitoring & Auditing
### Access Logs
```typescript
const logs = await client.conversations.list({
agent_id: 'agent_123',
from_date: '2025-01-01',
to_date: '2025-12-31'
});
```
### Compliance Reports
- Monthly conversation volume
- Data retention adherence
- Security incidents
- User consent rates
---
## Incident Response
### Data Breach Protocol
1. Identify affected conversations
2. Notify ElevenLabs immediately
3. Delete compromised data
4. Notify affected users (GDPR requirement)
5. Document incident
6. Review security controls
### Contact
security@elevenlabs.io

View File

@@ -0,0 +1,275 @@
# Cost Optimization Guide
## 1. LLM Caching
### How It Works
- **First request**: Full cost (`input_cache_write`)
- **Subsequent requests**: 10% cost (`input_cache_read`)
- **Cache TTL**: 5 minutes to 1 hour (configurable)
### Configuration
```json
{
"llm_config": {
"caching": {
"enabled": true,
"ttl_seconds": 3600
}
}
}
```
### What Gets Cached
✅ System prompts
✅ Tool definitions
✅ Knowledge base context
✅ Recent conversation history
❌ User messages (always fresh)
❌ Dynamic variables
❌ Tool responses
### Savings
**Up to 90%** on cached inputs
**Example**:
- System prompt: 500 tokens
- Without caching: 500 tokens × 100 conversations = 50,000 tokens
- With caching: 500 tokens (first) + 50 tokens × 99 (cached) = 5,450 tokens
- **Savings**: 89%
---
## 2. Model Swapping
### Model Comparison
| Model | Cost (per 1M tokens) | Speed | Quality | Best For |
|-------|---------------------|-------|---------|----------|
| GPT-4o | $5 | Medium | Highest | Complex reasoning |
| GPT-4o-mini | $0.15 | Fast | High | Most use cases |
| Claude Sonnet 4.5 | $3 | Medium | Highest | Long context |
| Gemini 2.5 Flash | $0.075 | Fastest | Medium | Simple tasks |
### Configuration
```json
{
"llm_config": {
"model": "gpt-4o-mini"
}
}
```
### Optimization Strategy
1. **Start with gpt-4o-mini** for all agents
2. **Upgrade to gpt-4o** only if:
- Complex reasoning required
- High accuracy critical
- User feedback indicates quality issues
3. **Use Gemini 2.5 Flash** for:
- Simple routing/classification
- FAQ responses
- Order status lookups
### Savings
**Up to 97%** (gpt-4o → Gemini 2.5 Flash)
---
## 3. Burst Pricing
### How It Works
- **Normal**: Your subscription concurrency limit (e.g., 10 calls)
- **Burst**: Up to 3× your limit (e.g., 30 calls)
- **Cost**: 2× per-minute rate for burst calls
### Configuration
```json
{
"call_limits": {
"burst_pricing_enabled": true
}
}
```
### When to Use
✅ Black Friday traffic spikes
✅ Product launches
✅ Seasonal demand (holidays)
✅ Marketing campaigns
❌ Sustained high traffic (upgrade plan instead)
❌ Unpredictable usage patterns
### Cost Calculation
**Example**:
- Subscription: 10 concurrent calls ($0.10/min per call)
- Traffic spike: 25 concurrent calls
- Burst calls: 25 - 10 = 15 calls
- Burst cost: 15 × $0.20/min = $3/min
- Regular cost: 10 × $0.10/min = $1/min
- **Total**: $4/min during spike
---
## 4. Prompt Optimization
### Reduce Token Count
**Before** (500 tokens):
```
You are a highly experienced and knowledgeable customer support specialist with extensive training in technical troubleshooting, customer service best practices, and empathetic communication. You should always maintain a professional yet friendly demeanor while helping customers resolve their issues efficiently and effectively.
```
**After** (150 tokens):
```
You are an experienced support specialist. Be professional, friendly, and efficient.
```
**Savings**: 70% token reduction
### Use Tools Instead of Context
**Before**: Include FAQ in system prompt (2,000 tokens)
**After**: Use RAG/knowledge base (100 tokens + retrieval)
**Savings**: 95% for large knowledge bases
---
## 5. Turn-Taking Optimization
### Impact on Cost
| Mode | Latency | LLM Calls | Cost Impact |
|------|---------|-----------|-------------|
| Eager | Low | More | Higher (more interruptions) |
| Normal | Medium | Medium | Balanced |
| Patient | High | Fewer | Lower (fewer interruptions) |
### Recommendation
Use **Patient** mode for cost-sensitive applications where speed is less critical.
---
## 6. Voice Settings
### Speed vs Cost
| Speed | TTS Cost | User Experience |
|-------|----------|-----------------|
| 0.7x | Higher (longer audio) | Slow |
| 1.0x | Baseline | Natural |
| 1.2x | Lower (shorter audio) | Fast |
### Recommendation
Use **1.1x speed** for slight cost savings without compromising experience.
---
## 7. Conversation Duration Limits
### Configuration
```json
{
"conversation": {
"max_duration_seconds": 300 // 5 minutes
}
}
```
### Use Cases
- FAQ bots (limit: 2-3 minutes)
- Order status (limit: 1 minute)
- Full support (limit: 10-15 minutes)
### Savings
Prevents unexpectedly long conversations.
---
## 8. Analytics-Driven Optimization
### Monitor Metrics
1. **Average conversation duration**
2. **LLM tokens per conversation**
3. **Tool call frequency**
4. **Resolution rate**
### Identify Issues
- Long conversations → improve prompts or add escalation
- High token count → enable caching or shorten prompts
- Low resolution rate → upgrade model or improve knowledge base
---
## 9. Cost Monitoring
### API Usage Tracking
```typescript
const usage = await client.analytics.getLLMUsage({
agent_id: 'agent_123',
from_date: '2025-11-01',
to_date: '2025-11-30'
});
console.log('Total tokens:', usage.total_tokens);
console.log('Cached tokens:', usage.cached_tokens);
console.log('Cost:', usage.total_cost);
```
### Set Budgets
```json
{
"cost_limits": {
"daily_budget_usd": 100,
"monthly_budget_usd": 2000
}
}
```
---
## 10. Cost Optimization Checklist
### Before Launch
- [ ] Enable LLM caching
- [ ] Use gpt-4o-mini (not gpt-4o)
- [ ] Optimize prompt length
- [ ] Set conversation duration limits
- [ ] Use RAG instead of large system prompts
- [ ] Configure burst pricing if needed
### During Operation
- [ ] Monitor LLM token usage weekly
- [ ] Review conversation analytics monthly
- [ ] Test cheaper models quarterly
- [ ] Optimize prompts based on analytics
- [ ] Review and remove unused tools
### Continuous Improvement
- [ ] A/B test cheaper models
- [ ] Analyze long conversations
- [ ] Improve resolution rates
- [ ] Reduce average conversation duration
- [ ] Increase cache hit rates
---
## Expected Savings
**Baseline Configuration**:
- Model: gpt-4o
- No caching
- Average prompt: 1,000 tokens
- Average conversation: 5 minutes
- Cost: ~$0.50/conversation
**Optimized Configuration**:
- Model: gpt-4o-mini
- Caching enabled
- Average prompt: 300 tokens
- Average conversation: 3 minutes
- Cost: ~$0.05/conversation
**Total Savings**: **90%** 🎉

View File

@@ -0,0 +1,276 @@
# System Prompt Engineering Guide
## 6-Component Framework
### 1. Personality
Define who the agent is.
**Template**:
```
You are [NAME], a [ROLE/PROFESSION] at [COMPANY].
You have [EXPERIENCE/BACKGROUND].
Your traits: [LIST PERSONALITY TRAITS].
```
**Example**:
```
You are Sarah, a patient and knowledgeable technical support specialist at TechCorp.
You have 7 years of experience helping customers troubleshoot software issues.
Your traits: patient, empathetic, detail-oriented, solution-focused.
```
### 2. Environment
Describe the communication context.
**Template**:
```
You're communicating via [CHANNEL: phone/chat/video].
Consider [ENVIRONMENTAL FACTORS].
Adapt your communication style to [CONTEXT].
```
**Example**:
```
You're speaking with customers over the phone.
Background noise and poor connections are common.
Speak clearly, use short sentences, and occasionally pause for emphasis.
```
### 3. Tone
Specify speech patterns and formality.
**Template**:
```
Tone: [FORMALITY LEVEL].
Language: [CONTRACTIONS/JARGON GUIDELINES].
Verbosity: [SENTENCE LENGTH, RESPONSE LENGTH].
Emotional Expression: [GUIDELINES].
```
**Example**:
```
Tone: Professional yet warm and approachable.
Language: Use contractions ("I'm", "let's") for natural conversation. Avoid technical jargon unless the customer uses it first.
Verbosity: Keep responses to 2-3 sentences. Ask one question at a time.
Emotional Expression: Express empathy with phrases like "I understand how frustrating that must be."
```
### 4. Goal
Define objectives and success criteria.
**Template**:
```
Primary Goal: [MAIN OBJECTIVE]
Secondary Goals:
- [SUPPORTING OBJECTIVE 1]
- [SUPPORTING OBJECTIVE 2]
Success Criteria:
- [MEASURABLE OUTCOME 1]
- [MEASURABLE OUTCOME 2]
```
**Example**:
```
Primary Goal: Resolve customer technical issues on the first call.
Secondary Goals:
- Verify customer identity securely
- Document issue details accurately
- Provide proactive tips to prevent future issues
Success Criteria:
- Customer verbally confirms their issue is resolved
- Issue documented in CRM system
- Customer satisfaction score ≥ 4/5
```
### 5. Guardrails
Set boundaries and ethical constraints.
**Template**:
```
Never:
- [PROHIBITED ACTION 1]
- [PROHIBITED ACTION 2]
Always:
- [REQUIRED ACTION 1]
- [REQUIRED ACTION 2]
Escalation Triggers:
- [CONDITION REQUIRING HUMAN INTERVENTION]
```
**Example**:
```
Never:
- Provide medical, legal, or financial advice
- Share confidential company information
- Make promises about refunds without verification
- Continue conversation if customer becomes abusive
Always:
- Verify customer identity before accessing account details
- Document all interactions in CRM
- Offer alternative solutions if first approach doesn't work
Escalation Triggers:
- Customer requests manager
- Issue requires account credit/refund approval
- Technical issue beyond your knowledge base
- Customer exhibits abusive behavior
```
### 6. Tools
Describe available functions and when to use them.
**Template**:
```
Available Tools:
1. tool_name(parameters)
Purpose: [WHAT IT DOES]
Use When: [TRIGGER CONDITION]
Example: [SAMPLE USAGE]
2. ...
Guidelines:
- [GENERAL TOOL USAGE RULES]
```
**Example**:
```
Available Tools:
1. lookup_order(order_id: string)
Purpose: Fetch order details from database
Use When: Customer mentions an order number or asks about order status
Example: "Let me look that up for you. [Call lookup_order(order_id='ORD-12345')]"
2. send_password_reset(email: string)
Purpose: Trigger password reset email
Use When: Customer can't access account and identity is verified
Example: "I'll send you a password reset email. [Call send_password_reset(email='customer@example.com')]"
3. transfer_to_supervisor()
Purpose: Escalate to human agent
Use When: Issue requires manager approval or customer explicitly requests
Example: "Let me connect you with a supervisor. [Call transfer_to_supervisor()]"
Guidelines:
- Always explain to the customer what you're doing before calling a tool
- Wait for tool response before continuing
- If tool fails, acknowledge and offer alternative
```
---
## Complete Example Templates
### Customer Support Agent
```
Personality:
You are Alex, a friendly and knowledgeable customer support specialist at TechCorp. You have 5 years of experience helping customers solve technical issues. You're patient, empathetic, and always maintain a positive attitude.
Environment:
You're speaking with customers over the phone. Communication is voice-only. Customers may have background noise or poor connection quality. Speak clearly and use thoughtful pauses for emphasis.
Tone:
Professional yet warm. Use contractions ("I'm", "let's") to sound natural. Avoid jargon unless the customer uses it first. Keep responses concise (2-3 sentences max). Use encouraging phrases like "I'll be happy to help with that."
Goal:
Primary: Resolve customer technical issues on the first call.
Secondary: Verify customer identity, document issues accurately, provide proactive solutions.
Success: Customer verbally confirms issue is resolved.
Guardrails:
- Never provide medical/legal/financial advice
- Don't share confidential company information
- Escalate if customer becomes abusive
- Never make promises about refunds without verification
Tools:
1. lookup_order(order_id) - Fetch order details when customer mentions order number
2. transfer_to_supervisor() - Escalate when issue requires manager approval
3. send_password_reset(email) - Trigger reset when customer can't access account
Always explain what you're doing before calling tools.
```
### Educational Tutor
```
Personality:
You are Maya, a patient and encouraging math tutor. You have 10 years of experience teaching middle school students. You're enthusiastic about learning and celebrate every small victory.
Environment:
You're tutoring students via voice chat. Students may feel anxious or frustrated about math. Create a safe, judgment-free environment where mistakes are learning opportunities.
Tone:
Warm, encouraging, and patient. Never sound frustrated or disappointed. Use positive reinforcement frequently ("Great thinking!", "You're on the right track!"). Adjust complexity based on student's responses.
Goal:
Primary: Help students understand math concepts, not just get answers.
Secondary: Build confidence and reduce math anxiety.
Success: Student can explain the concept in their own words and solve similar problems independently.
Guardrails:
- Never give answers directly—guide students to discover solutions
- Don't move to next topic until current concept is mastered
- If student becomes frustrated, take a break or switch to easier problem
- Never compare students or use negative language
Tools:
1. show_visual_aid(concept) - Display diagram or graph to illustrate concept
2. generate_practice_problem(difficulty) - Create custom practice problem
3. celebrate_achievement() - Play positive feedback animation
Always make learning feel like an achievement, not a chore.
```
---
## Prompt Engineering Tips
### Do's:
✅ Use specific examples in guidelines
✅ Define success criteria clearly
✅ Include escalation conditions
✅ Explain tool usage thoroughly
✅ Test prompts with real conversations
✅ Iterate based on analytics
### Don'ts:
❌ Use overly long prompts (increases cost)
❌ Be vague about goals or boundaries
❌ Include conflicting instructions
❌ Forget to test edge cases
❌ Use negative language excessively
❌ Overcomplicate simple tasks
---
## Testing Your Prompts
1. **Scenario Testing**: Run automated tests with success criteria
2. **Edge Case Testing**: Test boundary conditions and unusual inputs
3. **Tone Testing**: Evaluate conversation tone and empathy
4. **Tool Testing**: Verify tools are called correctly
5. **Analytics Review**: Monitor real conversations for issues
---
## Prompt Iteration Workflow
```
1. Write initial prompt using 6-component framework
2. Deploy to dev environment
3. Run 5-10 test conversations
4. Analyze transcripts for issues
5. Refine prompt based on findings
6. Deploy to staging
7. Run automated tests
8. Review analytics dashboard
9. Deploy to production
10. Monitor and iterate
```

188
references/testing-guide.md Normal file
View File

@@ -0,0 +1,188 @@
# Testing Guide
## 1. Scenario Testing (LLM-Based)
### Create Test
```bash
elevenlabs tests add "Refund Request" --template basic-llm
```
### Test Configuration
```json
{
"name": "Refund Request Test",
"scenario": "Customer requests refund for defective product",
"user_input": "I want a refund for order #12345. The product arrived broken.",
"success_criteria": [
"Agent acknowledges the issue empathetically",
"Agent asks for order number or uses provided number",
"Agent verifies order details",
"Agent provides clear next steps or refund timeline"
],
"evaluation_type": "llm"
}
```
### Run Test
```bash
elevenlabs agents test "Support Agent"
```
## 2. Tool Call Testing
### Test Configuration
```json
{
"name": "Order Lookup Test",
"scenario": "Customer asks about order status",
"user_input": "What's the status of order ORD-12345?",
"expected_tool_call": {
"tool_name": "lookup_order",
"parameters": {
"order_id": "ORD-12345"
}
}
}
```
## 3. Load Testing
### Basic Load Test
```bash
# 100 concurrent users, spawn 10/second, run for 5 minutes
elevenlabs test load \
--users 100 \
--spawn-rate 10 \
--duration 300
```
### With Burst Pricing
```json
{
"call_limits": {
"burst_pricing_enabled": true
}
}
```
## 4. Simulation API
### Programmatic Testing
```typescript
const simulation = await client.agents.simulate({
agent_id: 'agent_123',
scenario: 'Customer requests refund',
user_messages: [
"I want a refund for order #12345",
"It arrived broken",
"Yes, process the refund"
],
success_criteria: [
"Agent shows empathy",
"Agent verifies order",
"Agent provides timeline"
]
});
console.log('Passed:', simulation.passed);
console.log('Criteria met:', simulation.evaluation.criteria_met, '/', simulation.evaluation.criteria_total);
```
## 5. Convert Real Conversations to Tests
### From Dashboard
1. Navigate to Conversations
2. Select conversation
3. Click "Convert to Test"
4. Add success criteria
5. Save
### From API
```typescript
const test = await client.tests.createFromConversation({
conversation_id: 'conv_123',
success_criteria: [
"Issue was resolved",
"Customer satisfaction >= 4/5"
]
});
```
## 6. CI/CD Integration
### GitHub Actions
```yaml
name: Test Agent
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: Install CLI
run: npm install -g @elevenlabs/cli
- name: Push Tests
run: elevenlabs tests push
env:
ELEVENLABS_API_KEY: ${{ secrets.ELEVENLABS_API_KEY }}
- name: Run Tests
run: elevenlabs agents test "Support Agent"
env:
ELEVENLABS_API_KEY: ${{ secrets.ELEVENLABS_API_KEY }}
```
## 7. Test Organization
### Directory Structure
```
test_configs/
├── refund-tests/
│ ├── basic-refund.json
│ ├── duplicate-refund.json
│ └── expired-refund.json
├── order-lookup-tests/
│ ├── valid-order.json
│ └── invalid-order.json
└── escalation-tests/
├── angry-customer.json
└── complex-issue.json
```
## 8. Best Practices
### Do's:
✅ Test all conversation paths
✅ Include edge cases
✅ Test tool calls thoroughly
✅ Run tests before deployment
✅ Convert failed conversations to tests
✅ Monitor test trends over time
### Don'ts:
❌ Only test happy paths
❌ Ignore failing tests
❌ Skip load testing
❌ Test only in production
❌ Write vague success criteria
## 9. Metrics to Track
- **Pass Rate**: % of tests passing
- **Tool Accuracy**: % of correct tool calls
- **Response Time**: Average time to resolution
- **Load Capacity**: Max concurrent users before degradation
- **Error Rate**: % of conversations with errors
## 10. Debugging Failed Tests
1. Review conversation transcript
2. Check tool calls and parameters
3. Verify dynamic variables provided
4. Test prompt clarity
5. Check knowledge base content
6. Review guardrails and constraints
7. Iterate and retest

177
references/tool-examples.md Normal file
View File

@@ -0,0 +1,177 @@
# Tool Examples
## Client Tools (Browser-Side)
### Update Shopping Cart
```typescript
import { useConversation } from '@elevenlabs/react';
import { z } from 'zod';
clientTools: {
updateCart: {
description: "Add or remove items from the shopping cart",
parameters: z.object({
action: z.enum(['add', 'remove']),
item: z.string(),
quantity: z.number().min(1)
}),
handler: async ({ action, item, quantity }) => {
const cart = getCart();
if (action === 'add') {
cart.add(item, quantity);
} else {
cart.remove(item, quantity);
}
return { success: true, total: cart.total, items: cart.items.length };
}
}
}
```
### Navigate to Page
```typescript
navigate: {
description: "Navigate user to a different page",
parameters: z.object({
url: z.string().url()
}),
handler: async ({ url }) => {
window.location.href = url;
return { success: true };
}
}
```
## Server Tools (Webhooks)
### Get Weather
```json
{
"name": "get_weather",
"description": "Fetch current weather for a city",
"url": "https://api.weather.com/v1/current",
"method": "GET",
"parameters": {
"type": "object",
"properties": {
"city": { "type": "string", "description": "City name (e.g., 'London')" }
},
"required": ["city"]
},
"headers": {
"Authorization": "Bearer {{secret__weather_api_key}}"
}
}
```
### Stripe Payment
```json
{
"name": "create_payment_intent",
"description": "Create a Stripe payment intent for order",
"url": "https://api.stripe.com/v1/payment_intents",
"method": "POST",
"parameters": {
"type": "object",
"properties": {
"amount": { "type": "number", "description": "Amount in cents" },
"currency": { "type": "string", "description": "Currency code (e.g., 'usd')" }
},
"required": ["amount", "currency"]
},
"headers": {
"Authorization": "Bearer {{secret__stripe_api_key}}"
}
}
```
### CRM Integration
```json
{
"name": "update_crm",
"description": "Update customer record in CRM",
"url": "https://api.salesforce.com/services/data/v57.0/sobjects/Contact/{{contact_id}}",
"method": "PATCH",
"parameters": {
"type": "object",
"properties": {
"notes": { "type": "string" },
"status": { "type": "string", "enum": ["active", "resolved", "pending"] }
}
},
"headers": {
"Authorization": "Bearer {{secret__salesforce_token}}",
"Content-Type": "application/json"
}
}
```
## MCP Tools
### Connect PostgreSQL MCP Server
```json
{
"name": "PostgreSQL Database",
"server_url": "https://mcp.example.com/postgres",
"transport": "sse",
"secret_token": "{{secret__mcp_auth_token}}",
"approval_mode": "fine_grained"
}
```
### Connect File System MCP Server
```json
{
"name": "File System Access",
"server_url": "https://mcp.example.com/filesystem",
"transport": "http",
"approval_mode": "always_ask"
}
```
## System Tools
### Update Conversation State
```json
{
"name": "update_state",
"description": "Update conversation context",
"parameters": {
"key": { "type": "string" },
"value": { "type": "string" }
}
}
```
### Transfer to Human
```json
{
"name": "transfer_to_human",
"description": "Transfer call to human agent",
"parameters": {
"reason": { "type": "string", "description": "Reason for transfer" }
}
}
```
## Best Practices
**Client Tools**:
- Keep handler logic simple
- Always return meaningful values
- Handle errors gracefully
**Server Tools**:
- Use secret variables for API keys
- Provide clear parameter descriptions
- Include format examples in descriptions
**MCP Tools**:
- Test connectivity before production
- Use appropriate approval modes
- Monitor tool usage and errors
**System Tools**:
- Use for workflow state management
- Document state schema
- Clean up state when conversation ends

View File

@@ -0,0 +1,132 @@
# Workflow Examples
## Customer Support Routing
**Scenario**: Route calls to specialized agents based on customer needs.
```json
{
"workflow": {
"nodes": [
{
"id": "initial_routing",
"type": "subagent",
"config": {
"system_prompt": "Ask customer: Are you calling about billing, technical support, or sales?",
"turn_eagerness": "patient"
}
},
{
"id": "billing_agent",
"type": "subagent",
"config": {
"system_prompt": "You are a billing specialist. Help with invoices, payments, and account charges.",
"voice_id": "billing_voice_id"
}
},
{
"id": "technical_agent",
"type": "subagent",
"config": {
"system_prompt": "You are a technical support specialist. Troubleshoot product issues.",
"voice_id": "tech_voice_id"
}
},
{
"id": "sales_agent",
"type": "subagent",
"config": {
"system_prompt": "You are a sales representative. Help customers choose products.",
"voice_id": "sales_voice_id"
}
}
],
"edges": [
{ "from": "initial_routing", "to": "billing_agent", "condition": "user_mentions_billing" },
{ "from": "initial_routing", "to": "technical_agent", "condition": "user_mentions_technical" },
{ "from": "initial_routing", "to": "sales_agent", "condition": "user_mentions_sales" }
]
}
}
```
## Escalation Workflow
**Scenario**: Attempt self-service resolution, then escalate to human if needed.
```json
{
"workflow": {
"nodes": [
{
"id": "self_service",
"type": "subagent",
"config": {
"system_prompt": "Try to resolve issue using knowledge base and tools. If issue can't be resolved, offer human transfer.",
"knowledge_base": ["faq_doc_id"],
"tool_ids": ["lookup_order", "check_status"]
}
},
{
"id": "human_transfer",
"type": "tool",
"tool_name": "transfer_to_human"
}
],
"edges": [
{ "from": "self_service", "to": "human_transfer", "condition": "user_requests_human_or_issue_unresolved" }
]
}
}
```
## Multi-Language Support
**Scenario**: Detect language and route to appropriate voice/agent.
```json
{
"workflow": {
"nodes": [
{
"id": "language_detection",
"type": "subagent",
"config": {
"system_prompt": "Greet customer and detect language.",
"language": "auto"
}
},
{
"id": "english_agent",
"type": "subagent",
"config": {
"language": "en",
"voice_id": "en_voice_id",
"first_message": "Hello! How can I help you today?"
}
},
{
"id": "spanish_agent",
"type": "subagent",
"config": {
"language": "es",
"voice_id": "es_voice_id",
"first_message": "¡Hola! ¿Cómo puedo ayudarte hoy?"
}
}
],
"edges": [
{ "from": "language_detection", "to": "english_agent", "condition": "detected_language_en" },
{ "from": "language_detection", "to": "spanish_agent", "condition": "detected_language_es" }
]
}
}
```
## Best Practices
1. **Keep workflows simple** - Max 5-7 nodes for maintainability
2. **Test all paths** - Ensure every edge condition works
3. **Add fallbacks** - Always have a default path
4. **Monitor transitions** - Track which paths users take most
5. **Avoid loops** - Workflows can get stuck in infinite loops