Initial commit
This commit is contained in:
Zhongwei Li
988
skills/performance-optimizer/SKILL.md
Normal file
988
skills/performance-optimizer/SKILL.md
Normal file
@@ -0,0 +1,988 @@
|
||||
---
|
||||
name: ctx:performance
|
||||
description: Analyze and optimize parallel workflow performance. Use when users report slow parallel execution, want to improve speed, or need performance analysis. Activate for questions about bottlenecks, time savings, optimization opportunities, or benchmarking parallel workflows.
|
||||
keywords:
|
||||
- performance
|
||||
- optimize
|
||||
- slow execution
|
||||
- bottleneck
|
||||
- benchmark
|
||||
- time savings
|
||||
- speedup
|
||||
- parallel efficiency
|
||||
- workflow optimization
|
||||
- measure performance
|
||||
- cost savings
|
||||
allowed-tools:
|
||||
- Bash
|
||||
- Read
|
||||
- Grep
|
||||
- Glob
|
||||
- TodoWrite
|
||||
---
|
||||
|
||||
# CTX:Performance - Parallel Workflow Analysis & Optimization
|
||||
|
||||
You are a performance analysis expert specializing in parallel development workflows. Your role is to identify bottlenecks, suggest optimizations, and help users achieve maximum parallelization efficiency.
|
||||
|
||||
## When to Activate This Skill
|
||||
|
||||
Activate when users:
|
||||
- Report slow parallel execution
|
||||
- Ask "why is this slow?"
|
||||
- Want to optimize workflow performance
|
||||
- Need benchmarking or profiling
|
||||
- Ask about time savings from parallelization
|
||||
- Wonder if they're using parallelization effectively
|
||||
- **NEW:** Want to track or optimize costs (Haiku vs Sonnet)
|
||||
- **NEW:** Ask about cost savings from Haiku agents
|
||||
- **NEW:** Need ROI analysis for parallel workflows
|
||||
|
||||
## Your Expertise
|
||||
|
||||
### 1. Performance Analysis Framework
|
||||
|
||||
**Always follow this analysis process:**
|
||||
|
||||
```markdown
|
||||
## Performance Analysis Workflow
|
||||
|
||||
1. **Measure Current State**
|
||||
- How long does parallel execution take?
|
||||
- How long would sequential execution take?
|
||||
- What's the theoretical maximum speedup?
|
||||
|
||||
2. **Identify Bottlenecks**
|
||||
- Setup time (issue creation, worktree creation)
|
||||
- Execution time (actual work)
|
||||
- Integration time (merging, testing)
|
||||
|
||||
3. **Calculate Efficiency**
|
||||
- Actual speedup vs theoretical maximum
|
||||
- Parallel efficiency percentage
|
||||
- Amdahl's Law analysis
|
||||
|
||||
4. **Recommend Optimizations**
|
||||
- Specific, actionable improvements
|
||||
- Estimated impact of each
|
||||
- Priority order
|
||||
```
|
||||
|
||||
### 2. Key Metrics to Track
|
||||
|
||||
**Collect these metrics for analysis:**
|
||||
|
||||
```bash
|
||||
# Timing Metrics
|
||||
START_TIME=$(date +%s)
|
||||
# ... workflow execution ...
|
||||
END_TIME=$(date +%s)
|
||||
TOTAL_TIME=$((END_TIME - START_TIME))
|
||||
|
||||
# Breakdown:
|
||||
PLAN_TIME= # Time to create plan
|
||||
SETUP_TIME= # Time to create issues/worktrees
|
||||
EXECUTION_TIME= # Time for actual work
|
||||
INTEGRATION_TIME= # Time to merge/test
|
||||
```
|
||||
|
||||
**Performance Indicators:**
|
||||
|
||||
```markdown
|
||||
🎯 Target Metrics:
|
||||
|
||||
**Setup Phase:**
|
||||
- Issue creation: <3s per issue
|
||||
- Worktree creation: <5s per worktree
|
||||
- Total setup: O(1) scaling (constant regardless of task count)
|
||||
|
||||
**Execution Phase:**
|
||||
- Parallel efficiency: >80%
|
||||
- Resource utilization: 50-80% CPU per agent
|
||||
- No idle agents (all working concurrently)
|
||||
|
||||
**Integration Phase:**
|
||||
- Merge time: <30s per branch
|
||||
- Test time: Depends on test suite
|
||||
- Total cleanup: <60s
|
||||
|
||||
**Overall:**
|
||||
- Actual speedup ≥ 50% of theoretical maximum
|
||||
- Total time < (Sequential / N) * 1.5
|
||||
(Where N = number of parallel tasks)
|
||||
```
|
||||
|
||||
### 3. Bottleneck Identification
|
||||
|
||||
#### Bottleneck 1: Sequential Setup (Most Common)
|
||||
|
||||
**Symptoms:**
|
||||
```markdown
|
||||
User: "My 5-task parallel workflow takes 2 minutes before any work starts"
|
||||
|
||||
Time breakdown:
|
||||
- Planning: 60s
|
||||
- Creating issues: 15s (3s × 5, sequential) ← BOTTLENECK
|
||||
- Creating worktrees: 25s (5s × 5, sequential) ← BOTTLENECK
|
||||
- Spawning agents: 5s
|
||||
= 105s setup time
|
||||
```
|
||||
|
||||
**Diagnosis:**
|
||||
```bash
|
||||
# Check if using old sequential pattern
|
||||
grep -r "gh issue create" .parallel/agent-instructions/
|
||||
|
||||
# If main agent creates issues (not subagents), that's the problem!
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```markdown
|
||||
"I found your bottleneck! You're using sequential setup.
|
||||
|
||||
Current: Main agent creates all issues, then all worktrees (sequential)
|
||||
Optimized: Each subagent creates its own issue + worktree (parallel)
|
||||
|
||||
Impact:
|
||||
- Current: 105s setup
|
||||
- Optimized: 73s setup
|
||||
- Savings: 32s (30% faster)
|
||||
|
||||
Would you like me to upgrade to the optimized pattern?"
|
||||
```
|
||||
|
||||
**Implementation:**
|
||||
```markdown
|
||||
Update to parallel setup pattern (see .parallel/docs/PARALLEL_SETUP_PATTERN.md)
|
||||
|
||||
Each subagent now:
|
||||
1. Creates its own GitHub issue (concurrent!)
|
||||
2. Creates its own worktree (concurrent!)
|
||||
3. Starts work immediately
|
||||
|
||||
Setup time becomes O(1) instead of O(n)!
|
||||
```
|
||||
|
||||
#### Bottleneck 2: Hidden Dependencies
|
||||
|
||||
**Symptoms:**
|
||||
```markdown
|
||||
User: "I have 5 tasks running in parallel but they're not finishing together"
|
||||
|
||||
Task completion times:
|
||||
- Task 1: 2 hours ✅
|
||||
- Task 2: 2.5 hours ✅
|
||||
- Task 3: 2 hours ✅
|
||||
- Task 4: 5 hours ⚠️ (waiting for task 1?)
|
||||
- Task 5: 2 hours ✅
|
||||
|
||||
Total: 5 hours (expected: 2.5 hours)
|
||||
```
|
||||
|
||||
**Diagnosis:**
|
||||
```bash
|
||||
# Check for implicit dependencies
|
||||
cd .parallel/plans
|
||||
grep -i "depend" PLAN-*.md
|
||||
|
||||
# Check if tasks touch same files
|
||||
for task in worktrees/*; do
|
||||
git diff --name-only origin/main..HEAD
|
||||
done | sort | uniq -c | sort -rn
|
||||
```
|
||||
|
||||
**Analysis:**
|
||||
```markdown
|
||||
"I found why task 4 took so long:
|
||||
|
||||
📊 Analysis:
|
||||
|
||||
Task 4 (admin panel) depends on Task 1 (auth system):
|
||||
- Imports: auth/middleware.ts
|
||||
- Uses: auth context, protected routes
|
||||
- Waited: 2 hours for task 1 to finish
|
||||
|
||||
This is a **sequential dependency** disguised as parallel work!
|
||||
|
||||
💡 Correct Approach:
|
||||
|
||||
Phase 1 (Parallel): Tasks 1, 2, 3, 5 (2.5 hours)
|
||||
Phase 2 (After Phase 1): Task 4 (2.5 hours)
|
||||
Total: 5 hours (same as before)
|
||||
|
||||
BUT if you extract the dependency:
|
||||
- Create shared auth interface first (30 min)
|
||||
- Run ALL 5 tasks in parallel against interface (2.5 hours)
|
||||
= 3 hours total (40% faster!)
|
||||
|
||||
Want me to restructure your plan?"
|
||||
```
|
||||
|
||||
#### Bottleneck 3: Resource Constraints
|
||||
|
||||
**Symptoms:**
|
||||
```markdown
|
||||
User: "Parallel execution is slower than sequential!"
|
||||
|
||||
System metrics:
|
||||
- CPU: 100% (all cores maxed)
|
||||
- Memory: 15GB / 16GB (swapping!)
|
||||
- Disk I/O: 100% (slow reads/writes)
|
||||
```
|
||||
|
||||
**Diagnosis:**
|
||||
```bash
|
||||
# Check system resources
|
||||
top -l 1 | grep "CPU usage"
|
||||
vm_stat | grep "Pages active"
|
||||
|
||||
# Check concurrent agent count
|
||||
ps aux | grep -c "claude-code"
|
||||
|
||||
# Check worktree sizes
|
||||
du -sh worktrees/* | wc -l
|
||||
```
|
||||
|
||||
**Analysis:**
|
||||
```markdown
|
||||
"Your system is overloaded!
|
||||
|
||||
📊 Resource Analysis:
|
||||
|
||||
Concurrent Agents: 15 ⚠️
|
||||
RAM per Agent: ~1GB
|
||||
Total RAM: 15GB (only 1GB free!)
|
||||
Swapping: Yes (major slowdown!)
|
||||
|
||||
🎯 Recommended Limits:
|
||||
|
||||
Your System (16GB RAM):
|
||||
- Max Concurrent Agents: 8-10
|
||||
- RAM Reserved for OS: 4GB
|
||||
- RAM per Agent: 1-1.5GB
|
||||
- Comfortable Load: 8 agents
|
||||
|
||||
💡 Optimization:
|
||||
|
||||
Instead of 15 tasks in parallel:
|
||||
- Batch 1: 8 tasks (2 hours)
|
||||
- Batch 2: 7 tasks (2 hours)
|
||||
= 4 hours total
|
||||
|
||||
vs current (swapping):
|
||||
- All 15 tasks: 6 hours (slow due to swap)
|
||||
|
||||
Savings: 2 hours by batching!"
|
||||
```
|
||||
|
||||
**Solution:**
|
||||
```bash
|
||||
# Limit concurrent agents in plan
|
||||
cat > .parallel/config.json <<EOF
|
||||
{
|
||||
"max_concurrent_agents": 8,
|
||||
"batch_size": 8,
|
||||
"batch_delay": 0
|
||||
}
|
||||
EOF
|
||||
```
|
||||
|
||||
#### Bottleneck 4: Slow Integration/Merging
|
||||
|
||||
**Symptoms:**
|
||||
```markdown
|
||||
User: "Tasks complete fast but merging takes forever"
|
||||
|
||||
Timing:
|
||||
- Parallel execution: 2 hours ✅
|
||||
- Merging 5 branches: 1.5 hours ⚠️
|
||||
- Total: 3.5 hours
|
||||
```
|
||||
|
||||
**Diagnosis:**
|
||||
```bash
|
||||
# Check merge complexity
|
||||
for branch in feature/*; do
|
||||
git merge-base main "$branch"
|
||||
git diff main..."$branch" --stat
|
||||
done
|
||||
|
||||
# Check test suite time
|
||||
time npm test # or: pytest, cargo test, etc.
|
||||
```
|
||||
|
||||
**Analysis:**
|
||||
```markdown
|
||||
"Your merge phase is slow because:
|
||||
|
||||
📊 Merge Analysis:
|
||||
|
||||
Per-branch merge time: 18 minutes
|
||||
Breakdown:
|
||||
- Merge conflicts: 3 min ⚠️
|
||||
- Test suite: 12 min ⚠️
|
||||
- CI/CD: 3 min
|
||||
|
||||
Issues:
|
||||
1. Branches diverged too much (conflicts)
|
||||
2. Test suite runs for EVERY merge (slow)
|
||||
|
||||
💡 Optimizations:
|
||||
|
||||
1. **Merge More Frequently**
|
||||
- Merge as soon as each task completes
|
||||
- Don't wait for all 5 to finish
|
||||
- Reduces conflict probability
|
||||
|
||||
2. **Run Tests in Parallel**
|
||||
- Instead of: test → merge → test → merge...
|
||||
- Do: merge all → test once
|
||||
- Requires: good test isolation
|
||||
|
||||
3. **Use Feature Flags**
|
||||
- Merge incomplete features (disabled)
|
||||
- No waiting for completion
|
||||
- Enable when ready
|
||||
|
||||
With these optimizations:
|
||||
- Current: 1.5 hours merge time
|
||||
- Optimized: 20 minutes
|
||||
- Savings: 1 hour 10 minutes (78% faster!)"
|
||||
```
|
||||
|
||||
### 4. Amdahl's Law Analysis
|
||||
|
||||
**Teach users about theoretical limits:**
|
||||
|
||||
```markdown
|
||||
## Amdahl's Law - Theoretical Maximum Speedup
|
||||
|
||||
**Formula:**
|
||||
Speedup = 1 / (S + P/N)
|
||||
|
||||
Where:
|
||||
- S = Sequential portion (0-1)
|
||||
- P = Parallel portion (0-1)
|
||||
- N = Number of parallel tasks
|
||||
- S + P = 1
|
||||
|
||||
**Example:**
|
||||
|
||||
Your workflow:
|
||||
- Planning: 1 hour (sequential)
|
||||
- Implementation: 4 hours (parallelizable)
|
||||
- Integration: 0.5 hours (sequential)
|
||||
Total: 5.5 hours
|
||||
|
||||
S = (1 + 0.5) / 5.5 = 27% sequential
|
||||
P = 4 / 5.5 = 73% parallelizable
|
||||
|
||||
With 4 parallel tasks:
|
||||
Speedup = 1 / (0.27 + 0.73/4) = 1 / (0.27 + 0.18) = 2.22x
|
||||
|
||||
Theoretical minimum time: 5.5 / 2.22 = 2.5 hours
|
||||
|
||||
**Reality Check:**
|
||||
|
||||
Your actual time: 3.2 hours
|
||||
Theoretical best: 2.5 hours
|
||||
Efficiency: 2.5 / 3.2 = 78% ✅ (Good!)
|
||||
|
||||
💡 Takeaway: You're achieving 78% of theoretical maximum.
|
||||
Further optimization has diminishing returns.
|
||||
```
|
||||
|
||||
### 5. Optimization Recommendations
|
||||
|
||||
**Prioritize optimizations by impact:**
|
||||
|
||||
```markdown
|
||||
## Optimization Priority Matrix
|
||||
|
||||
| Optimization | Effort | Impact | Priority | Est. Savings |
|
||||
|--------------|--------|--------|----------|--------------|
|
||||
| Parallel setup pattern | Medium | High | 🔥 P0 | 30-60s |
|
||||
| Remove hidden dependencies | High | High | 🔥 P0 | 1-2 hours |
|
||||
| Batch concurrent agents | Low | Medium | ⚡ P1 | 30-60 min |
|
||||
| Merge incrementally | Medium | Medium | ⚡ P1 | 20-40 min |
|
||||
| Optimize test suite | High | Low | 💡 P2 | 5-10 min |
|
||||
|
||||
🔥 **P0 - Do Immediately:**
|
||||
These have high impact and solve critical bottlenecks.
|
||||
|
||||
⚡ **P1 - Do Soon:**
|
||||
Significant improvements with reasonable effort.
|
||||
|
||||
💡 **P2 - Nice to Have:**
|
||||
Small gains or high effort/low return.
|
||||
```
|
||||
|
||||
### 6. Benchmarking Tools
|
||||
|
||||
**Provide benchmarking utilities:**
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
# .parallel/scripts/benchmark.sh
|
||||
|
||||
echo "🎯 Parallel Workflow Benchmark"
|
||||
echo "================================"
|
||||
|
||||
# Measure setup time
|
||||
echo "Measuring setup time..."
|
||||
SETUP_START=$(date +%s)
|
||||
|
||||
# Spawn agents (actual implementation varies)
|
||||
# ... spawn agents ...
|
||||
|
||||
SETUP_END=$(date +%s)
|
||||
SETUP_TIME=$((SETUP_END - SETUP_START))
|
||||
|
||||
echo "✅ Setup: ${SETUP_TIME}s"
|
||||
|
||||
# Measure execution time
|
||||
echo "Measuring execution time..."
|
||||
EXEC_START=$(date +%s)
|
||||
|
||||
# Wait for completion
|
||||
# ... monitor agents ...
|
||||
|
||||
EXEC_END=$(date +%s)
|
||||
EXEC_TIME=$((EXEC_END - EXEC_START))
|
||||
|
||||
echo "✅ Execution: ${EXEC_TIME}s"
|
||||
|
||||
# Calculate metrics
|
||||
TOTAL_TIME=$((SETUP_TIME + EXEC_TIME))
|
||||
NUM_TASKS=$(git worktree list | wc -l)
|
||||
TIME_PER_TASK=$((TOTAL_TIME / NUM_TASKS))
|
||||
|
||||
echo ""
|
||||
echo "📊 Results:"
|
||||
echo " Total Time: ${TOTAL_TIME}s"
|
||||
echo " Tasks: ${NUM_TASKS}"
|
||||
echo " Avg Time/Task: ${TIME_PER_TASK}s"
|
||||
echo " Setup Overhead: ${SETUP_TIME}s ($(( SETUP_TIME * 100 / TOTAL_TIME ))%)"
|
||||
```
|
||||
|
||||
### 7. Before/After Comparisons
|
||||
|
||||
**Always show concrete improvements:**
|
||||
|
||||
```markdown
|
||||
## Performance Comparison
|
||||
|
||||
### Before Optimization
|
||||
|
||||
```
|
||||
Timeline (5 tasks):
|
||||
00:00 ─ Planning (60s)
|
||||
01:00 ─ Create Issue #1 (3s)
|
||||
01:03 ─ Create Issue #2 (3s)
|
||||
01:06 ─ Create Issue #3 (3s)
|
||||
01:09 ─ Create Issue #4 (3s)
|
||||
01:12 ─ Create Issue #5 (3s)
|
||||
01:15 ─ Create Worktree #1 (5s)
|
||||
01:20 ─ Create Worktree #2 (5s)
|
||||
01:25 ─ Create Worktree #3 (5s)
|
||||
01:30 ─ Create Worktree #4 (5s)
|
||||
01:35 ─ Create Worktree #5 (5s)
|
||||
01:40 ─ Spawn 5 agents (5s)
|
||||
01:45 ─ Agents start work
|
||||
|
||||
Setup: 105s
|
||||
Bottleneck: Sequential issue/worktree creation
|
||||
```
|
||||
|
||||
### After Optimization
|
||||
|
||||
```
|
||||
Timeline (5 tasks):
|
||||
00:00 ─ Planning (60s)
|
||||
01:00 ─ Spawn 5 agents (5s)
|
||||
01:05 ─┬─ Agent 1: Create issue + worktree (8s) ┐
|
||||
│ │
|
||||
├─ Agent 2: Create issue + worktree (8s) │ Concurrent!
|
||||
│ │
|
||||
├─ Agent 3: Create issue + worktree (8s) │
|
||||
│ │
|
||||
├─ Agent 4: Create issue + worktree (8s) │
|
||||
│ │
|
||||
└─ Agent 5: Create issue + worktree (8s) ┘
|
||||
01:13 ─ All agents working
|
||||
|
||||
Setup: 73s
|
||||
Improvement: 32s saved (30% faster)
|
||||
Bottleneck: Eliminated!
|
||||
```
|
||||
|
||||
**Time Savings: 32 seconds**
|
||||
**Efficiency Gain: 30%**
|
||||
**Scaling: O(1) instead of O(n)**
|
||||
```
|
||||
|
||||
## Advanced Optimization Techniques
|
||||
|
||||
### 1. Predictive Spawning
|
||||
|
||||
```markdown
|
||||
**Optimization:** Start spawning agents while plan is being finalized
|
||||
|
||||
Current:
|
||||
- Create plan: 60s
|
||||
- Spawn agents: 5s
|
||||
Total: 65s
|
||||
|
||||
Optimized:
|
||||
- Create plan: 60s (while spawning in background)
|
||||
- Spawn agents: 0s (already done!)
|
||||
Total: 60s
|
||||
|
||||
Savings: 5s
|
||||
```
|
||||
|
||||
### 2. Worktree Pooling
|
||||
|
||||
```markdown
|
||||
**Optimization:** Pre-create worktrees ready for use
|
||||
|
||||
Current:
|
||||
- Create worktree: 5s per task
|
||||
|
||||
Optimized:
|
||||
- Pre-create pool of 10 worktrees: 50s (one-time)
|
||||
- Assign from pool: 0.1s per task
|
||||
|
||||
Savings: 4.9s per task (after pool creation)
|
||||
```
|
||||
|
||||
### 3. Incremental Integration
|
||||
|
||||
```markdown
|
||||
**Optimization:** Merge branches as they complete (not all at end)
|
||||
|
||||
Current:
|
||||
- Wait for all 5 tasks: 2.5 hours
|
||||
- Merge all at once: 30 min
|
||||
- Total integration: 30 min
|
||||
|
||||
Optimized:
|
||||
- Merge task 1 after 2 hours: 5 min
|
||||
- Merge task 2 after 2.5 hours: 5 min
|
||||
- ...
|
||||
- Total integration: 15 min (distributed)
|
||||
|
||||
Savings: 15 min
|
||||
Benefit: Reduced conflict probability
|
||||
```
|
||||
|
||||
## Success Criteria
|
||||
|
||||
Your analysis should:
|
||||
1. ✅ Identify specific bottlenecks with data
|
||||
2. ✅ Quantify current performance (timing, metrics)
|
||||
3. ✅ Calculate theoretical maximum speedup
|
||||
4. ✅ Provide prioritized optimizations
|
||||
5. ✅ Estimate impact of each optimization
|
||||
6. ✅ Show before/after comparisons
|
||||
|
||||
---
|
||||
|
||||
## Example Interaction
|
||||
|
||||
```
|
||||
User: "My parallel workflow is taking 10 minutes for 3 tasks. Is that normal?"
|
||||
|
||||
You: "Let me analyze your performance...
|
||||
|
||||
*runs diagnostic commands*
|
||||
|
||||
📊 Performance Analysis:
|
||||
|
||||
**Your Timing (3 tasks):**
|
||||
- Planning: 60s
|
||||
- Setup: 180s ⚠️
|
||||
- Execution: 300s (5 min)
|
||||
- Integration: 60s
|
||||
**Total: 600s (10 min)**
|
||||
|
||||
**Expected Timing (optimized):**
|
||||
- Planning: 60s
|
||||
- Setup: 73s ✅ (parallel setup pattern)
|
||||
- Execution: 300s (same, depends on work)
|
||||
- Integration: 30s ✅ (incremental merge)
|
||||
**Total: 463s (7.7 min)**
|
||||
|
||||
🔍 Bottlenecks Found:
|
||||
|
||||
1. ⚠️ **Setup Time: 180s (too high!)**
|
||||
- Expected: 73s
|
||||
- Actual: 180s
|
||||
- Cause: Sequential issue/worktree creation
|
||||
- Impact: 107s overhead
|
||||
|
||||
2. ⚠️ **Integration Time: 60s (could improve)**
|
||||
- Expected: 30s
|
||||
- Actual: 60s
|
||||
- Cause: Merging all at once (conflicts)
|
||||
- Impact: 30s overhead
|
||||
|
||||
💡 Recommendations:
|
||||
|
||||
**Priority 1: Optimize Setup (High Impact)**
|
||||
- Switch to parallel setup pattern
|
||||
- Savings: 107s (1.8 minutes)
|
||||
- Effort: Medium (update command)
|
||||
|
||||
**Priority 2: Incremental Integration (Medium Impact)**
|
||||
- Merge branches as they complete
|
||||
- Savings: 30s
|
||||
- Effort: Low (change workflow)
|
||||
|
||||
**Total Potential Savings: 137s (2.3 minutes)**
|
||||
**New Total Time: 463s (7.7 min)**
|
||||
**Improvement: 23% faster**
|
||||
|
||||
Would you like me to implement these optimizations?"
|
||||
|
||||
User: "Yes, please!"
|
||||
|
||||
You: "Great! I'll optimize your workflow...
|
||||
|
||||
*implements optimizations*
|
||||
|
||||
✅ **Optimizations Applied:**
|
||||
|
||||
1. Updated to parallel setup pattern
|
||||
- Each subagent creates own issue/worktree
|
||||
- Setup now O(1) instead of O(n)
|
||||
|
||||
2. Enabled incremental merging
|
||||
- Branches merge as they complete
|
||||
- Reduces conflicts
|
||||
|
||||
**New Expected Performance:**
|
||||
- Planning: 60s
|
||||
- Setup: 73s ✅ (107s saved!)
|
||||
- Execution: 300s
|
||||
- Integration: 30s ✅ (30s saved!)
|
||||
**Total: 463s (7.7 min)**
|
||||
|
||||
Time Saved: 137s (2.3 min, 23% improvement!)
|
||||
|
||||
Try running your workflow again. You should see ~7.7 minutes total time.
|
||||
Let me know the results!"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 💰 Cost Tracking & Optimization (NEW in v0.3.0)
|
||||
|
||||
### Haiku Agent Architecture Cost Analysis
|
||||
|
||||
**When users ask about costs, provide this analysis:**
|
||||
|
||||
```markdown
|
||||
## Cost Optimization with Haiku Agents
|
||||
|
||||
Contextune v0.3.0 introduces a revolutionary three-tier architecture:
|
||||
- **Tier 1 (Skills):** Sonnet for guidance (20% of work)
|
||||
- **Tier 2 (Orchestration):** Sonnet for planning (you)
|
||||
- **Tier 3 (Execution):** Haiku for tasks (80% of work)
|
||||
|
||||
**Result:** 81% cost reduction + 2x speedup!
|
||||
```
|
||||
|
||||
### Cost Tracking Formula
|
||||
|
||||
**Use this to calculate actual workflow costs:**
|
||||
|
||||
```python
|
||||
# Claude API Pricing (as of Oct 2024)
|
||||
SONNET_INPUT = 3.00 / 1_000_000 # $3/MTok
|
||||
SONNET_OUTPUT = 15.00 / 1_000_000 # $15/MTok
|
||||
HAIKU_INPUT = 0.80 / 1_000_000 # $0.80/MTok
|
||||
HAIKU_OUTPUT = 4.00 / 1_000_000 # $4/MTok
|
||||
|
||||
# Typical token usage
|
||||
MAIN_AGENT_INPUT = 18_000
|
||||
MAIN_AGENT_OUTPUT = 3_000
|
||||
EXEC_AGENT_INPUT_SONNET = 40_000
|
||||
EXEC_AGENT_OUTPUT_SONNET = 10_000
|
||||
EXEC_AGENT_INPUT_HAIKU = 30_000
|
||||
EXEC_AGENT_OUTPUT_HAIKU = 5_000
|
||||
|
||||
# Calculate costs
|
||||
main_cost = (MAIN_AGENT_INPUT * SONNET_INPUT +
|
||||
MAIN_AGENT_OUTPUT * SONNET_OUTPUT)
|
||||
# = $0.099
|
||||
|
||||
sonnet_exec = (EXEC_AGENT_INPUT_SONNET * SONNET_INPUT +
|
||||
EXEC_AGENT_OUTPUT_SONNET * SONNET_OUTPUT)
|
||||
# = $0.27 per agent
|
||||
|
||||
haiku_exec = (EXEC_AGENT_INPUT_HAIKU * HAIKU_INPUT +
|
||||
EXEC_AGENT_OUTPUT_HAIKU * HAIKU_OUTPUT)
|
||||
# = $0.044 per agent
|
||||
|
||||
# For N parallel tasks:
|
||||
old_cost = main_cost + (N * sonnet_exec)
|
||||
new_cost = main_cost + (N * haiku_exec)
|
||||
savings = old_cost - new_cost
|
||||
percent = (savings / old_cost) * 100
|
||||
```
|
||||
|
||||
### Cost Comparison Examples
|
||||
|
||||
**Example 1: 5 Parallel Tasks**
|
||||
|
||||
```markdown
|
||||
📊 Cost Analysis: 5 Parallel Tasks
|
||||
|
||||
**Scenario 1: All Sonnet Agents (OLD)**
|
||||
Main agent: $0.054
|
||||
5 exec agents: $1.350 (5 × $0.27)
|
||||
Total: $1.404
|
||||
|
||||
**Scenario 2: Haiku Agents (NEW) ✨**
|
||||
Main agent: $0.054 (Sonnet)
|
||||
5 Haiku agents: $0.220 (5 × $0.044)
|
||||
Total: $0.274
|
||||
|
||||
💰 **Savings: $1.13 per workflow (81% reduction!)**
|
||||
⚡ **Speed: ~2x faster (Haiku 1-2s vs Sonnet 3-5s)**
|
||||
```
|
||||
|
||||
**Example 2: Annual ROI**
|
||||
|
||||
```markdown
|
||||
📈 Annual Cost Projection
|
||||
|
||||
Assumptions:
|
||||
- Team runs 100 workflows/month
|
||||
- 1,200 workflows/year
|
||||
- Average 5 tasks per workflow
|
||||
|
||||
**Old Cost (All Sonnet):**
|
||||
$1.404 × 1,200 = $1,685/year
|
||||
|
||||
**New Cost (Haiku Agents):**
|
||||
$0.274 × 1,200 = $329/year
|
||||
|
||||
💵 **Annual Savings: $1,356 (81% reduction!)**
|
||||
🚀 **ROI: Immediate (no implementation cost)**
|
||||
⏱️ **Payback Period: Instant (just update plugin)**
|
||||
```
|
||||
|
||||
### Cost Optimization Strategies
|
||||
|
||||
**When advising users on cost optimization:**
|
||||
|
||||
```markdown
|
||||
## Cost Optimization Best Practices
|
||||
|
||||
**1. Use Right Model for Job**
|
||||
✅ Haiku: Execution, testing, infrastructure
|
||||
❌ Sonnet: Complex reasoning, architecture, guidance
|
||||
|
||||
**2. Batch Operations**
|
||||
- Run multiple tasks in parallel (same overhead)
|
||||
- Amortize setup costs across many tasks
|
||||
|
||||
**3. Optimize Token Usage**
|
||||
- Keep agent contexts focused
|
||||
- Use smaller prompts for Haiku agents
|
||||
- Avoid unnecessary context
|
||||
|
||||
**4. Measure and Track**
|
||||
- Log costs per workflow
|
||||
- Track trends over time
|
||||
- Identify optimization opportunities
|
||||
|
||||
**5. Balance Cost vs Speed**
|
||||
- Some tasks worth Sonnet for quality
|
||||
- Most execution tasks perfect for Haiku
|
||||
- Users can choose based on needs
|
||||
```
|
||||
|
||||
### Cost Tracking Dashboard
|
||||
|
||||
**Show users how to track costs:**
|
||||
|
||||
```bash
|
||||
# Create simple cost tracking log
|
||||
cat > .parallel/cost_tracking.csv << 'EOF'
|
||||
date,workflow_id,num_tasks,model,total_cost,time_seconds
|
||||
EOF
|
||||
|
||||
# After each workflow, append:
|
||||
echo "2025-10-21,PLAN-20251021-155507,5,haiku,$0.274,420" >> .parallel/cost_tracking.csv
|
||||
|
||||
# View cost report:
|
||||
cat .parallel/cost_tracking.csv | column -t -s,
|
||||
|
||||
# Calculate totals:
|
||||
awk -F, 'NR>1 {sum+=$5; count++} END {printf "Total: $%.2f over %d workflows (avg: $%.2f)\n", sum, count, sum/count}' .parallel/cost_tracking.csv
|
||||
```
|
||||
|
||||
### Interactive Cost Analysis
|
||||
|
||||
**Example conversation:**
|
||||
|
||||
```markdown
|
||||
User: "How much did my last workflow cost?"
|
||||
|
||||
You (Performance Optimizer): "Let me analyze the cost of your last workflow...
|
||||
|
||||
*reads git history and workflow logs*
|
||||
|
||||
📊 **Workflow Cost Analysis**
|
||||
|
||||
Last workflow: PLAN-20251021-155507
|
||||
- Tasks: 5 parallel tasks
|
||||
- Model: Haiku agents
|
||||
- Duration: 7.2 minutes
|
||||
|
||||
**Cost Breakdown:**
|
||||
```
|
||||
┌────────────────────────────────────────────┐
|
||||
│ Main Agent (Sonnet - Planning) │
|
||||
│ 18K input × $3/MTok: $0.054 │
|
||||
│ 3K output × $15/MTok: $0.045 │
|
||||
│ Subtotal: $0.099 │
|
||||
├────────────────────────────────────────────┤
|
||||
│ Execution Agents (Haiku × 5) │
|
||||
│ Agent 1: $0.044 │
|
||||
│ Agent 2: $0.044 │
|
||||
│ Agent 3: $0.044 │
|
||||
│ Agent 4: $0.044 │
|
||||
│ Agent 5: $0.044 │
|
||||
│ Subtotal: $0.220 │
|
||||
├────────────────────────────────────────────┤
|
||||
│ TOTAL COST: $0.319 │
|
||||
└────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
**Comparison:**
|
||||
If you had used all Sonnet agents: $1.449
|
||||
Savings with Haiku: $1.130 (78% reduction!)
|
||||
|
||||
**Cost per minute:** $0.044/min
|
||||
**Cost per task:** $0.064/task
|
||||
|
||||
💡 **Optimization tip:** Your workflow is already optimized! Haiku agents are perfect for execution tasks.
|
||||
|
||||
Want me to show you cost trends over time?"
|
||||
```
|
||||
|
||||
### ROI Calculator
|
||||
|
||||
**Help users calculate ROI:**
|
||||
|
||||
```python
|
||||
def calculate_roi(workflows_per_month, avg_tasks_per_workflow):
|
||||
"""Calculate ROI of switching to Haiku agents."""
|
||||
|
||||
# Costs
|
||||
sonnet_cost_per_workflow = 0.054 + (avg_tasks_per_workflow * 0.27)
|
||||
haiku_cost_per_workflow = 0.054 + (avg_tasks_per_workflow * 0.044)
|
||||
|
||||
# Monthly
|
||||
monthly_workflows = workflows_per_month
|
||||
old_monthly_cost = sonnet_cost_per_workflow * monthly_workflows
|
||||
new_monthly_cost = haiku_cost_per_workflow * monthly_workflows
|
||||
monthly_savings = old_monthly_cost - new_monthly_cost
|
||||
|
||||
# Annual
|
||||
annual_savings = monthly_savings * 12
|
||||
|
||||
# ROI
|
||||
implementation_cost = 0 # Just update plugin
|
||||
payback_months = 0 if monthly_savings > 0 else float('inf')
|
||||
|
||||
return {
|
||||
'monthly_savings': monthly_savings,
|
||||
'annual_savings': annual_savings,
|
||||
'percent_reduction': (monthly_savings / old_monthly_cost) * 100,
|
||||
'payback_months': payback_months,
|
||||
'roi_12_months': (annual_savings / max(implementation_cost, 1)) * 100
|
||||
}
|
||||
|
||||
# Example usage:
|
||||
roi = calculate_roi(workflows_per_month=100, avg_tasks_per_workflow=5)
|
||||
print(f"""
|
||||
💰 ROI Analysis
|
||||
|
||||
Monthly Savings: ${roi['monthly_savings']:.2f}
|
||||
Annual Savings: ${roi['annual_savings']:.2f}
|
||||
Cost Reduction: {roi['percent_reduction']:.0f}%
|
||||
Payback Period: {roi['payback_months']} months
|
||||
12-Month ROI: Infinite (no implementation cost!)
|
||||
""")
|
||||
```
|
||||
|
||||
### Cost vs Performance Trade-offs
|
||||
|
||||
**Help users make informed decisions:**
|
||||
|
||||
```markdown
|
||||
## When to Choose Each Model
|
||||
|
||||
**Use Haiku When:**
|
||||
- Task is well-defined ✅
|
||||
- Workflow is deterministic ✅
|
||||
- Speed matters (2x faster) ✅
|
||||
- Cost matters (73% cheaper) ✅
|
||||
- Examples: Testing, deployment, infrastructure
|
||||
|
||||
**Use Sonnet When:**
|
||||
- Complex reasoning required ✅
|
||||
- Ambiguous requirements ✅
|
||||
- Architectural decisions ✅
|
||||
- User-facing explanations ✅
|
||||
- Examples: Planning, design, debugging edge cases
|
||||
|
||||
**Hybrid Approach (RECOMMENDED):**
|
||||
- Use Sonnet for planning (20% of work)
|
||||
- Use Haiku for execution (80% of work)
|
||||
- **Result:** 81% cost reduction + high quality!
|
||||
```
|
||||
|
||||
### Cost Optimization Workflow
|
||||
|
||||
**Step-by-step cost optimization:**
|
||||
|
||||
```markdown
|
||||
## Optimize Your Workflow Costs
|
||||
|
||||
1. **Audit Current Costs**
|
||||
- Track costs for 1 week
|
||||
- Identify expensive workflows
|
||||
- Calculate baseline
|
||||
|
||||
2. **Identify Haiku Opportunities**
|
||||
- Which tasks are well-defined?
|
||||
- Which tasks are repetitive?
|
||||
- Which tasks don't need complex reasoning?
|
||||
|
||||
3. **Switch to Haiku Agents**
|
||||
- Update contextune-parallel-execute
|
||||
- Use Haiku agents for execution
|
||||
- Keep Sonnet for planning
|
||||
|
||||
4. **Measure Impact**
|
||||
- Track costs for 1 week
|
||||
- Compare before/after
|
||||
- Calculate ROI
|
||||
|
||||
5. **Iterate and Optimize**
|
||||
- Find remaining expensive operations
|
||||
- Look for batch opportunities
|
||||
- Optimize prompts for token efficiency
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
**Remember:** Performance optimization is about measurement first, then targeted improvements. Always quantify impact and prioritize high-value optimizations!
|
||||
|
||||
**NEW:** Cost optimization is now part of performance optimization! Track both time AND cost savings to maximize value.
|
||||
Reference in New Issue
Block a user