Initial commit
This commit is contained in:
89
skills/troubleshoot/SKILL.md
Normal file
89
skills/troubleshoot/SKILL.md
Normal file
@@ -0,0 +1,89 @@
|
||||
---
|
||||
name: troubleshoot
|
||||
description: Diagnose and fix claude-mem installation issues. Checks PM2 worker status, database integrity, service health, dependencies, and provides automated fixes for common problems.
|
||||
---
|
||||
|
||||
# Claude-Mem Troubleshooting Skill
|
||||
|
||||
Diagnose and resolve installation and operational issues with the claude-mem plugin.
|
||||
|
||||
## When to Use This Skill
|
||||
|
||||
**Invoke this skill when:**
|
||||
- Memory not persisting after `/clear`
|
||||
- Viewer UI empty or not loading
|
||||
- Worker service not running
|
||||
- Database missing or corrupted
|
||||
- Port conflicts
|
||||
- Missing dependencies
|
||||
- "Nothing is remembered" complaints
|
||||
- Search results empty when they shouldn't be
|
||||
|
||||
**Do NOT invoke** for feature requests or usage questions (use regular documentation for that).
|
||||
|
||||
## Quick Decision Guide
|
||||
|
||||
Once the skill is loaded, choose the appropriate operation:
|
||||
|
||||
**What's the problem?**
|
||||
|
||||
- "Nothing is being remembered" → [operations/common-issues.md](operations/common-issues.md#nothing-remembered)
|
||||
- "Viewer is empty" → [operations/common-issues.md](operations/common-issues.md#viewer-empty)
|
||||
- "Worker won't start" → [operations/common-issues.md](operations/common-issues.md#worker-not-starting)
|
||||
- "Want to run full diagnostics" → [operations/diagnostics.md](operations/diagnostics.md)
|
||||
- "Need automated fix" → [operations/automated-fixes.md](operations/automated-fixes.md)
|
||||
|
||||
## Available Operations
|
||||
|
||||
Choose the appropriate operation file for detailed instructions:
|
||||
|
||||
### Diagnostic Workflows
|
||||
1. **[Full System Diagnostics](operations/diagnostics.md)** - Comprehensive step-by-step diagnostic workflow
|
||||
2. **[Worker Diagnostics](operations/worker.md)** - PM2 worker-specific troubleshooting
|
||||
3. **[Database Diagnostics](operations/database.md)** - Database integrity and data checks
|
||||
|
||||
### Issue Resolution
|
||||
4. **[Common Issues](operations/common-issues.md)** - Quick fixes for frequently encountered problems
|
||||
5. **[Automated Fixes](operations/automated-fixes.md)** - One-command fix sequences
|
||||
|
||||
### Reference
|
||||
6. **[Quick Commands](operations/reference.md)** - Essential commands for troubleshooting
|
||||
|
||||
## Quick Start
|
||||
|
||||
**Fast automated fix (try this first):**
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/ && \
|
||||
pm2 delete claude-mem-worker 2>/dev/null; \
|
||||
npm install && \
|
||||
node_modules/.bin/pm2 start ecosystem.config.cjs && \
|
||||
sleep 3 && \
|
||||
curl -s http://127.0.0.1:37777/health
|
||||
```
|
||||
|
||||
Expected output: `{"status":"ok"}`
|
||||
|
||||
If that doesn't work, proceed to detailed diagnostics.
|
||||
|
||||
## Response Format
|
||||
|
||||
When troubleshooting:
|
||||
1. **Identify the symptom** - What's the user reporting?
|
||||
2. **Choose operation file** - Use the decision guide above
|
||||
3. **Follow steps systematically** - Don't skip diagnostic steps
|
||||
4. **Report findings** - Tell user what you found and what was fixed
|
||||
5. **Verify resolution** - Confirm the issue is resolved
|
||||
|
||||
## Technical Notes
|
||||
|
||||
- **Worker port:** Default 37777 (configurable via `CLAUDE_MEM_WORKER_PORT`)
|
||||
- **Database location:** `~/.claude-mem/claude-mem.db`
|
||||
- **Plugin location:** `~/.claude/plugins/marketplaces/thedotmack/`
|
||||
- **PM2 process name:** `claude-mem-worker`
|
||||
|
||||
## Error Reporting
|
||||
|
||||
If troubleshooting doesn't resolve the issue, collect diagnostic data and direct user to:
|
||||
https://github.com/thedotmack/claude-mem/issues
|
||||
|
||||
See [operations/diagnostics.md](operations/diagnostics.md#reporting-issues) for details on what to collect.
|
||||
151
skills/troubleshoot/operations/automated-fixes.md
Normal file
151
skills/troubleshoot/operations/automated-fixes.md
Normal file
@@ -0,0 +1,151 @@
|
||||
# Automated Fix Sequences
|
||||
|
||||
One-command fix sequences for common claude-mem issues.
|
||||
|
||||
## Quick Fix: Complete Reset and Restart
|
||||
|
||||
**Use when:** General issues, worker not responding, after updates
|
||||
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/ && \
|
||||
pm2 delete claude-mem-worker 2>/dev/null; \
|
||||
npm install && \
|
||||
node_modules/.bin/pm2 start ecosystem.config.cjs && \
|
||||
sleep 3 && \
|
||||
curl -s http://127.0.0.1:37777/health
|
||||
```
|
||||
|
||||
**Expected output:** `{"status":"ok"}`
|
||||
|
||||
**What it does:**
|
||||
1. Stops the worker (if running)
|
||||
2. Ensures dependencies are installed
|
||||
3. Starts worker with local PM2
|
||||
4. Waits for startup
|
||||
5. Verifies health
|
||||
|
||||
## Fix: Worker Not Running
|
||||
|
||||
**Use when:** PM2 shows worker as stopped or not listed
|
||||
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/ && \
|
||||
node_modules/.bin/pm2 start ecosystem.config.cjs && \
|
||||
sleep 2 && \
|
||||
pm2 status
|
||||
```
|
||||
|
||||
**Expected output:** Worker shows as "online"
|
||||
|
||||
## Fix: Dependencies Missing
|
||||
|
||||
**Use when:** Worker won't start due to missing packages
|
||||
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/ && \
|
||||
npm install && \
|
||||
pm2 restart claude-mem-worker
|
||||
```
|
||||
|
||||
## Fix: Port Conflict
|
||||
|
||||
**Use when:** Error shows port already in use
|
||||
|
||||
```bash
|
||||
# Change to port 37778
|
||||
mkdir -p ~/.claude-mem && \
|
||||
echo '{"env":{"CLAUDE_MEM_WORKER_PORT":"37778"}}' > ~/.claude-mem/settings.json && \
|
||||
pm2 restart claude-mem-worker && \
|
||||
sleep 2 && \
|
||||
curl -s http://127.0.0.1:37778/health
|
||||
```
|
||||
|
||||
**Expected output:** `{"status":"ok"}`
|
||||
|
||||
## Fix: Database Issues
|
||||
|
||||
**Use when:** Database appears corrupted or out of sync
|
||||
|
||||
```bash
|
||||
# Backup and test integrity
|
||||
cp ~/.claude-mem/claude-mem.db ~/.claude-mem/claude-mem.db.backup && \
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "PRAGMA integrity_check;" && \
|
||||
pm2 restart claude-mem-worker
|
||||
```
|
||||
|
||||
**If integrity check fails, recreate database:**
|
||||
```bash
|
||||
# WARNING: This deletes all memory data
|
||||
mv ~/.claude-mem/claude-mem.db ~/.claude-mem/claude-mem.db.old && \
|
||||
pm2 restart claude-mem-worker
|
||||
```
|
||||
|
||||
## Fix: Clean Reinstall
|
||||
|
||||
**Use when:** All else fails, nuclear option
|
||||
|
||||
```bash
|
||||
# Backup data first
|
||||
cp ~/.claude-mem/claude-mem.db ~/.claude-mem/claude-mem.db.backup 2>/dev/null
|
||||
|
||||
# Stop and remove worker
|
||||
pm2 delete claude-mem-worker 2>/dev/null
|
||||
|
||||
# Reinstall dependencies
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/ && \
|
||||
rm -rf node_modules && \
|
||||
npm install
|
||||
|
||||
# Start worker
|
||||
node_modules/.bin/pm2 start ecosystem.config.cjs && \
|
||||
sleep 3 && \
|
||||
curl -s http://127.0.0.1:37777/health
|
||||
```
|
||||
|
||||
## Fix: Clear PM2 Logs
|
||||
|
||||
**Use when:** Logs are too large, want fresh start
|
||||
|
||||
```bash
|
||||
pm2 flush claude-mem-worker && \
|
||||
pm2 restart claude-mem-worker
|
||||
```
|
||||
|
||||
## Verification Commands
|
||||
|
||||
**After running any fix, verify with these:**
|
||||
|
||||
```bash
|
||||
# Check worker status
|
||||
pm2 status | grep claude-mem-worker
|
||||
|
||||
# Check health
|
||||
curl -s http://127.0.0.1:37777/health
|
||||
|
||||
# Check database
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations;"
|
||||
|
||||
# Check viewer
|
||||
curl -s http://127.0.0.1:37777/api/stats
|
||||
|
||||
# Check logs for errors
|
||||
pm2 logs claude-mem-worker --lines 20 --nostream | grep -i error
|
||||
```
|
||||
|
||||
**All checks should pass:**
|
||||
- Worker status: "online"
|
||||
- Health: `{"status":"ok"}`
|
||||
- Database: Shows count (may be 0 if new)
|
||||
- Stats: Returns JSON with counts
|
||||
- Logs: No recent errors
|
||||
|
||||
## Troubleshooting the Fixes
|
||||
|
||||
**If automated fix fails:**
|
||||
1. Run the diagnostic script from [diagnostics.md](diagnostics.md)
|
||||
2. Check specific error in PM2 logs
|
||||
3. Try manual worker start to see detailed error:
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
node plugin/scripts/worker-service.cjs
|
||||
```
|
||||
232
skills/troubleshoot/operations/common-issues.md
Normal file
232
skills/troubleshoot/operations/common-issues.md
Normal file
@@ -0,0 +1,232 @@
|
||||
# Common Issue Resolutions
|
||||
|
||||
Quick fixes for frequently encountered claude-mem problems.
|
||||
|
||||
## Issue: Nothing is Remembered After `/clear` {#nothing-remembered}
|
||||
|
||||
**Symptoms:**
|
||||
- Data doesn't persist across sessions
|
||||
- Context is empty after `/clear`
|
||||
- Search returns no results for past work
|
||||
|
||||
**Root cause:** Sessions are marked complete but data should persist. This suggests:
|
||||
- Worker not processing observations
|
||||
- Database not being written to
|
||||
- Context hook not reading from database
|
||||
|
||||
**Fix:**
|
||||
1. Verify worker is running:
|
||||
```bash
|
||||
pm2 jlist | grep claude-mem-worker
|
||||
```
|
||||
|
||||
2. Check database has recent observations:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations WHERE created_at > datetime('now', '-1 day');"
|
||||
```
|
||||
|
||||
3. Restart worker and start new session:
|
||||
```bash
|
||||
pm2 restart claude-mem-worker
|
||||
```
|
||||
|
||||
4. Create a test observation: `/skill version-bump` then cancel
|
||||
|
||||
5. Check if observation appears in viewer:
|
||||
```bash
|
||||
open http://127.0.0.1:37777
|
||||
# Or manually check database:
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT * FROM observations ORDER BY created_at DESC LIMIT 1;"
|
||||
```
|
||||
|
||||
## Issue: Viewer Empty After Every Claude Restart {#viewer-empty}
|
||||
|
||||
**Symptoms:**
|
||||
- Viewer shows no data at http://127.0.0.1:37777
|
||||
- Stats endpoint returns all zeros
|
||||
- Database appears empty in UI
|
||||
|
||||
**Root cause:**
|
||||
- Database being recreated on startup (shouldn't happen)
|
||||
- Worker reading from wrong database location
|
||||
- Database permissions issue
|
||||
|
||||
**Fix:**
|
||||
1. Check database file exists and has data:
|
||||
```bash
|
||||
ls -lh ~/.claude-mem/claude-mem.db
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations;"
|
||||
```
|
||||
|
||||
2. Check file permissions:
|
||||
```bash
|
||||
ls -la ~/.claude-mem/claude-mem.db
|
||||
# Should be readable/writable by your user
|
||||
```
|
||||
|
||||
3. Verify worker is using correct database path in logs:
|
||||
```bash
|
||||
pm2 logs claude-mem-worker --lines 50 --nostream | grep "Database"
|
||||
```
|
||||
|
||||
4. Test viewer connection manually:
|
||||
```bash
|
||||
curl -s http://127.0.0.1:37777/api/stats
|
||||
# Should show non-zero counts if data exists
|
||||
```
|
||||
|
||||
## Issue: Old Memory in Claude {#old-memory}
|
||||
|
||||
**Symptoms:**
|
||||
- Context contains outdated observations
|
||||
- Irrelevant past work appearing in sessions
|
||||
- Context feels stale
|
||||
|
||||
**Root cause:** Context hook injecting stale observations
|
||||
|
||||
**Fix:**
|
||||
1. Check the observation count setting:
|
||||
```bash
|
||||
grep CLAUDE_MEM_CONTEXT_OBSERVATIONS ~/.claude/settings.json
|
||||
```
|
||||
|
||||
2. Default is 50 observations - you can adjust this:
|
||||
```json
|
||||
{
|
||||
"env": {
|
||||
"CLAUDE_MEM_CONTEXT_OBSERVATIONS": "25"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. Check database for actual observation dates:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT created_at, project, title FROM observations ORDER BY created_at DESC LIMIT 10;"
|
||||
```
|
||||
|
||||
4. Consider filtering by project if working on multiple codebases
|
||||
|
||||
## Issue: Worker Not Starting {#worker-not-starting}
|
||||
|
||||
**Symptoms:**
|
||||
- PM2 shows worker as "stopped" or "errored"
|
||||
- Health check fails
|
||||
- Viewer not accessible
|
||||
|
||||
**Root cause:**
|
||||
- Port already in use
|
||||
- PM2 not installed or not in PATH
|
||||
- Missing dependencies
|
||||
|
||||
**Fix:**
|
||||
1. Try manual worker start to see error:
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
node plugin/scripts/worker-service.cjs
|
||||
# Should start server on port 37777 or show error
|
||||
```
|
||||
|
||||
2. If port in use, change it:
|
||||
```bash
|
||||
mkdir -p ~/.claude-mem
|
||||
echo '{"env":{"CLAUDE_MEM_WORKER_PORT":"37778"}}' > ~/.claude-mem/settings.json
|
||||
```
|
||||
|
||||
3. If dependencies missing:
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
npm install
|
||||
pm2 start ecosystem.config.cjs
|
||||
```
|
||||
|
||||
## Issue: Search Results Empty
|
||||
|
||||
**Symptoms:**
|
||||
- Search skill returns no results
|
||||
- API endpoints return empty arrays
|
||||
- Know there's data but can't find it
|
||||
|
||||
**Root cause:**
|
||||
- FTS5 tables not synchronized
|
||||
- Wrong project filter
|
||||
- Database not being queried correctly
|
||||
|
||||
**Fix:**
|
||||
1. Check if observations exist in database:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations;"
|
||||
```
|
||||
|
||||
2. Check FTS5 table sync:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations_fts;"
|
||||
# Should match observation count
|
||||
```
|
||||
|
||||
3. Try search via API directly:
|
||||
```bash
|
||||
curl "http://127.0.0.1:37777/api/search/observations?q=test&format=index"
|
||||
```
|
||||
|
||||
4. If FTS5 out of sync, restart worker (triggers reindex):
|
||||
```bash
|
||||
pm2 restart claude-mem-worker
|
||||
```
|
||||
|
||||
## Issue: Port Conflicts
|
||||
|
||||
**Symptoms:**
|
||||
- Worker won't start
|
||||
- Error: "EADDRINUSE: address already in use"
|
||||
- Health check fails
|
||||
|
||||
**Fix:**
|
||||
1. Check what's using port 37777:
|
||||
```bash
|
||||
lsof -i :37777
|
||||
```
|
||||
|
||||
2. Either kill the conflicting process or change claude-mem port:
|
||||
```bash
|
||||
mkdir -p ~/.claude-mem
|
||||
echo '{"env":{"CLAUDE_MEM_WORKER_PORT":"37778"}}' > ~/.claude-mem/settings.json
|
||||
pm2 restart claude-mem-worker
|
||||
```
|
||||
|
||||
## Issue: Database Corrupted
|
||||
|
||||
**Symptoms:**
|
||||
- SQLite errors in logs
|
||||
- Worker crashes on startup
|
||||
- Queries fail
|
||||
|
||||
**Fix:**
|
||||
1. Backup the database:
|
||||
```bash
|
||||
cp ~/.claude-mem/claude-mem.db ~/.claude-mem/claude-mem.db.backup
|
||||
```
|
||||
|
||||
2. Try to repair:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "PRAGMA integrity_check;"
|
||||
```
|
||||
|
||||
3. If repair fails, recreate (loses data):
|
||||
```bash
|
||||
rm ~/.claude-mem/claude-mem.db
|
||||
pm2 restart claude-mem-worker
|
||||
# Worker will create new database
|
||||
```
|
||||
|
||||
## Prevention Tips
|
||||
|
||||
**Keep claude-mem healthy:**
|
||||
- Regularly check viewer UI to see if observations are being captured
|
||||
- Monitor database size (shouldn't grow unbounded)
|
||||
- Update plugin when new versions are released
|
||||
- Keep Claude Code updated
|
||||
|
||||
**Performance tuning:**
|
||||
- Adjust `CLAUDE_MEM_CONTEXT_OBSERVATIONS` if context is too large/small
|
||||
- Use `/clear` to mark sessions complete and start fresh
|
||||
- Use search skill to query specific memories instead of loading everything
|
||||
403
skills/troubleshoot/operations/database.md
Normal file
403
skills/troubleshoot/operations/database.md
Normal file
@@ -0,0 +1,403 @@
|
||||
# Database Diagnostics
|
||||
|
||||
SQLite database troubleshooting for claude-mem.
|
||||
|
||||
## Database Overview
|
||||
|
||||
Claude-mem uses SQLite3 for persistent storage:
|
||||
- **Location:** `~/.claude-mem/claude-mem.db`
|
||||
- **Library:** better-sqlite3 (synchronous, not bun:sqlite)
|
||||
- **Features:** FTS5 full-text search, triggers, indexes
|
||||
- **Tables:** observations, sessions, user_prompts, observations_fts, sessions_fts, prompts_fts
|
||||
|
||||
## Basic Database Checks
|
||||
|
||||
### Check Database Exists
|
||||
|
||||
```bash
|
||||
# Check file exists
|
||||
ls -lh ~/.claude-mem/claude-mem.db
|
||||
|
||||
# Check file size
|
||||
du -h ~/.claude-mem/claude-mem.db
|
||||
|
||||
# Check permissions
|
||||
ls -la ~/.claude-mem/claude-mem.db
|
||||
```
|
||||
|
||||
**Expected:**
|
||||
- File exists
|
||||
- Size: 100KB - 10MB+ (depends on usage)
|
||||
- Permissions: Readable/writable by your user
|
||||
|
||||
### Check Database Integrity
|
||||
|
||||
```bash
|
||||
# Run integrity check
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "PRAGMA integrity_check;"
|
||||
```
|
||||
|
||||
**Expected output:** `ok`
|
||||
|
||||
**If errors appear:**
|
||||
- Database corrupted
|
||||
- Backup immediately: `cp ~/.claude-mem/claude-mem.db ~/.claude-mem/claude-mem.db.backup`
|
||||
- Consider recreating (data loss)
|
||||
|
||||
## Data Inspection
|
||||
|
||||
### Count Records
|
||||
|
||||
```bash
|
||||
# Observation count
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations;"
|
||||
|
||||
# Session count
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM sessions;"
|
||||
|
||||
# User prompt count
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM user_prompts;"
|
||||
|
||||
# FTS5 table counts (should match main tables)
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations_fts;"
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM sessions_fts;"
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM prompts_fts;"
|
||||
```
|
||||
|
||||
### View Recent Records
|
||||
|
||||
```bash
|
||||
# Recent observations
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT
|
||||
created_at,
|
||||
type,
|
||||
title,
|
||||
project
|
||||
FROM observations
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 10;
|
||||
"
|
||||
|
||||
# Recent sessions
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT
|
||||
created_at,
|
||||
request,
|
||||
project
|
||||
FROM sessions
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 5;
|
||||
"
|
||||
|
||||
# Recent user prompts
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT
|
||||
created_at,
|
||||
prompt
|
||||
FROM user_prompts
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 10;
|
||||
"
|
||||
```
|
||||
|
||||
### Check Projects
|
||||
|
||||
```bash
|
||||
# List all projects
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT DISTINCT project
|
||||
FROM observations
|
||||
ORDER BY project;
|
||||
"
|
||||
|
||||
# Count observations per project
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT
|
||||
project,
|
||||
COUNT(*) as count
|
||||
FROM observations
|
||||
GROUP BY project
|
||||
ORDER BY count DESC;
|
||||
"
|
||||
```
|
||||
|
||||
## Database Schema
|
||||
|
||||
### View Table Structure
|
||||
|
||||
```bash
|
||||
# List all tables
|
||||
sqlite3 ~/.claude-mem/claude-mem.db ".tables"
|
||||
|
||||
# Show observations table schema
|
||||
sqlite3 ~/.claude-mem/claude-mem.db ".schema observations"
|
||||
|
||||
# Show all schemas
|
||||
sqlite3 ~/.claude-mem/claude-mem.db ".schema"
|
||||
```
|
||||
|
||||
### Expected Tables
|
||||
|
||||
- `observations` - Main observation records
|
||||
- `observations_fts` - FTS5 virtual table for full-text search
|
||||
- `sessions` - Session summary records
|
||||
- `sessions_fts` - FTS5 virtual table for session search
|
||||
- `user_prompts` - User prompt records
|
||||
- `prompts_fts` - FTS5 virtual table for prompt search
|
||||
|
||||
## FTS5 Synchronization
|
||||
|
||||
The FTS5 tables should stay synchronized with main tables via triggers.
|
||||
|
||||
### Check FTS5 Sync
|
||||
|
||||
```bash
|
||||
# Compare counts
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT
|
||||
(SELECT COUNT(*) FROM observations) as observations,
|
||||
(SELECT COUNT(*) FROM observations_fts) as observations_fts,
|
||||
(SELECT COUNT(*) FROM sessions) as sessions,
|
||||
(SELECT COUNT(*) FROM sessions_fts) as sessions_fts,
|
||||
(SELECT COUNT(*) FROM user_prompts) as prompts,
|
||||
(SELECT COUNT(*) FROM prompts_fts) as prompts_fts;
|
||||
"
|
||||
```
|
||||
|
||||
**Expected:** All pairs should match (observations = observations_fts, etc.)
|
||||
|
||||
### Fix FTS5 Desync
|
||||
|
||||
If FTS5 counts don't match, triggers may have failed. Restart worker to rebuild:
|
||||
|
||||
```bash
|
||||
pm2 restart claude-mem-worker
|
||||
```
|
||||
|
||||
The worker will rebuild FTS5 indexes on startup if they're out of sync.
|
||||
|
||||
## Common Database Issues
|
||||
|
||||
### Issue: Database Doesn't Exist
|
||||
|
||||
**Cause:** First run, or database was deleted
|
||||
|
||||
**Fix:** Database will be created automatically on first observation. No action needed.
|
||||
|
||||
### Issue: Database is Empty (0 Records)
|
||||
|
||||
**Cause:**
|
||||
- New installation (normal)
|
||||
- Data was deleted
|
||||
- Worker not processing observations
|
||||
|
||||
**Fix:**
|
||||
1. Create test observation (use any skill and cancel)
|
||||
2. Check worker logs for errors:
|
||||
```bash
|
||||
pm2 logs claude-mem-worker --lines 50 --nostream
|
||||
```
|
||||
3. Verify observation appears in database
|
||||
|
||||
### Issue: Database Permission Denied
|
||||
|
||||
**Cause:** File permissions wrong, database owned by different user
|
||||
|
||||
**Fix:**
|
||||
```bash
|
||||
# Check ownership
|
||||
ls -la ~/.claude-mem/claude-mem.db
|
||||
|
||||
# Fix permissions (if needed)
|
||||
chmod 644 ~/.claude-mem/claude-mem.db
|
||||
chown $USER ~/.claude-mem/claude-mem.db
|
||||
```
|
||||
|
||||
### Issue: Database Locked
|
||||
|
||||
**Cause:**
|
||||
- Multiple processes accessing database
|
||||
- Crash left lock file
|
||||
- Long-running transaction
|
||||
|
||||
**Fix:**
|
||||
```bash
|
||||
# Check for lock file
|
||||
ls -la ~/.claude-mem/claude-mem.db-wal
|
||||
ls -la ~/.claude-mem/claude-mem.db-shm
|
||||
|
||||
# Remove lock files (only if worker is stopped!)
|
||||
pm2 stop claude-mem-worker
|
||||
rm ~/.claude-mem/claude-mem.db-wal ~/.claude-mem/claude-mem.db-shm
|
||||
pm2 start claude-mem-worker
|
||||
```
|
||||
|
||||
### Issue: Database Growing Too Large
|
||||
|
||||
**Cause:** Too many observations accumulated
|
||||
|
||||
**Check size:**
|
||||
```bash
|
||||
du -h ~/.claude-mem/claude-mem.db
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations;"
|
||||
```
|
||||
|
||||
**Options:**
|
||||
1. Delete old observations (manual cleanup):
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
DELETE FROM observations
|
||||
WHERE created_at < datetime('now', '-90 days');
|
||||
"
|
||||
```
|
||||
|
||||
2. Vacuum to reclaim space:
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "VACUUM;"
|
||||
```
|
||||
|
||||
3. Archive and start fresh:
|
||||
```bash
|
||||
mv ~/.claude-mem/claude-mem.db ~/.claude-mem/claude-mem.db.archive
|
||||
pm2 restart claude-mem-worker
|
||||
```
|
||||
|
||||
## Database Recovery
|
||||
|
||||
### Backup Database
|
||||
|
||||
**Before any destructive operations:**
|
||||
```bash
|
||||
cp ~/.claude-mem/claude-mem.db ~/.claude-mem/claude-mem.db.backup
|
||||
```
|
||||
|
||||
### Restore from Backup
|
||||
|
||||
```bash
|
||||
pm2 stop claude-mem-worker
|
||||
cp ~/.claude-mem/claude-mem.db.backup ~/.claude-mem/claude-mem.db
|
||||
pm2 start claude-mem-worker
|
||||
```
|
||||
|
||||
### Export Data
|
||||
|
||||
Export to JSON for safekeeping:
|
||||
|
||||
```bash
|
||||
# Export observations
|
||||
sqlite3 ~/.claude-mem/claude-mem.db -json "SELECT * FROM observations;" > observations.json
|
||||
|
||||
# Export sessions
|
||||
sqlite3 ~/.claude-mem/claude-mem.db -json "SELECT * FROM sessions;" > sessions.json
|
||||
|
||||
# Export prompts
|
||||
sqlite3 ~/.claude-mem/claude-mem.db -json "SELECT * FROM user_prompts;" > prompts.json
|
||||
```
|
||||
|
||||
### Recreate Database
|
||||
|
||||
**WARNING: Data loss. Backup first!**
|
||||
|
||||
```bash
|
||||
# Stop worker
|
||||
pm2 stop claude-mem-worker
|
||||
|
||||
# Backup current database
|
||||
cp ~/.claude-mem/claude-mem.db ~/.claude-mem/claude-mem.db.old
|
||||
|
||||
# Delete database
|
||||
rm ~/.claude-mem/claude-mem.db
|
||||
|
||||
# Start worker (creates new database)
|
||||
pm2 start claude-mem-worker
|
||||
```
|
||||
|
||||
## Database Statistics
|
||||
|
||||
### Storage Analysis
|
||||
|
||||
```bash
|
||||
# Database file size
|
||||
du -h ~/.claude-mem/claude-mem.db
|
||||
|
||||
# Record counts by type
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT
|
||||
type,
|
||||
COUNT(*) as count
|
||||
FROM observations
|
||||
GROUP BY type
|
||||
ORDER BY count DESC;
|
||||
"
|
||||
|
||||
# Observations per month
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT
|
||||
strftime('%Y-%m', created_at) as month,
|
||||
COUNT(*) as count
|
||||
FROM observations
|
||||
GROUP BY month
|
||||
ORDER BY month DESC;
|
||||
"
|
||||
|
||||
# Average observation size (characters)
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT
|
||||
AVG(LENGTH(content)) as avg_content_length,
|
||||
MAX(LENGTH(content)) as max_content_length
|
||||
FROM observations;
|
||||
"
|
||||
```
|
||||
|
||||
## Advanced Queries
|
||||
|
||||
### Find Specific Observations
|
||||
|
||||
```bash
|
||||
# Search by keyword (FTS5)
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT title, created_at
|
||||
FROM observations_fts
|
||||
WHERE observations_fts MATCH 'authentication'
|
||||
ORDER BY created_at DESC;
|
||||
"
|
||||
|
||||
# Find by type
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT title, created_at
|
||||
FROM observations
|
||||
WHERE type = 'bugfix'
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 10;
|
||||
"
|
||||
|
||||
# Find by file path
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
SELECT title, created_at
|
||||
FROM observations
|
||||
WHERE file_path LIKE '%auth%'
|
||||
ORDER BY created_at DESC;
|
||||
"
|
||||
```
|
||||
|
||||
## Database Maintenance
|
||||
|
||||
### Regular Maintenance Tasks
|
||||
|
||||
```bash
|
||||
# Analyze for query optimization
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "ANALYZE;"
|
||||
|
||||
# Rebuild FTS5 indexes
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "
|
||||
INSERT INTO observations_fts(observations_fts) VALUES('rebuild');
|
||||
INSERT INTO sessions_fts(sessions_fts) VALUES('rebuild');
|
||||
INSERT INTO prompts_fts(prompts_fts) VALUES('rebuild');
|
||||
"
|
||||
|
||||
# Vacuum to reclaim space
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "VACUUM;"
|
||||
```
|
||||
|
||||
**Run monthly to keep database healthy.**
|
||||
219
skills/troubleshoot/operations/diagnostics.md
Normal file
219
skills/troubleshoot/operations/diagnostics.md
Normal file
@@ -0,0 +1,219 @@
|
||||
# Full System Diagnostics
|
||||
|
||||
Comprehensive step-by-step diagnostic workflow for claude-mem issues.
|
||||
|
||||
## Diagnostic Workflow
|
||||
|
||||
Run these checks systematically to identify the root cause:
|
||||
|
||||
### 1. Check PM2 Worker Status
|
||||
|
||||
First, verify if the worker service is running:
|
||||
|
||||
```bash
|
||||
# Check if PM2 is available
|
||||
which pm2 || echo "PM2 not found in PATH"
|
||||
|
||||
# List PM2 processes
|
||||
pm2 jlist 2>&1
|
||||
|
||||
# If pm2 is not found, try the local installation
|
||||
~/.claude/plugins/marketplaces/thedotmack/node_modules/.bin/pm2 jlist 2>&1
|
||||
```
|
||||
|
||||
**Expected output:** JSON array with `claude-mem-worker` process showing `"status": "online"`
|
||||
|
||||
**If worker not running or status is not "online":**
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
pm2 start ecosystem.config.cjs
|
||||
# Or use local pm2:
|
||||
node_modules/.bin/pm2 start ecosystem.config.cjs
|
||||
```
|
||||
|
||||
### 2. Check Worker Service Health
|
||||
|
||||
Test if the worker service responds to HTTP requests:
|
||||
|
||||
```bash
|
||||
# Default port is 37777
|
||||
curl -s http://127.0.0.1:37777/health
|
||||
|
||||
# Check custom port from settings
|
||||
PORT=$(cat ~/.claude-mem/settings.json 2>/dev/null | grep CLAUDE_MEM_WORKER_PORT | grep -o '[0-9]\+' || echo "37777")
|
||||
curl -s http://127.0.0.1:$PORT/health
|
||||
```
|
||||
|
||||
**Expected output:** `{"status":"ok"}`
|
||||
|
||||
**If connection refused:**
|
||||
- Worker not running → Go back to step 1
|
||||
- Port conflict → Check what's using the port:
|
||||
```bash
|
||||
lsof -i :37777 || netstat -tlnp | grep 37777
|
||||
```
|
||||
|
||||
### 3. Check Database
|
||||
|
||||
Verify the database exists and contains data:
|
||||
|
||||
```bash
|
||||
# Check if database file exists
|
||||
ls -lh ~/.claude-mem/claude-mem.db
|
||||
|
||||
# Check database size (should be > 0 bytes)
|
||||
du -h ~/.claude-mem/claude-mem.db
|
||||
|
||||
# Query database for observation count (requires sqlite3)
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) as observation_count FROM observations;" 2>&1
|
||||
|
||||
# Query for session count
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) as session_count FROM sessions;" 2>&1
|
||||
|
||||
# Check recent observations
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT created_at, type, title FROM observations ORDER BY created_at DESC LIMIT 5;" 2>&1
|
||||
```
|
||||
|
||||
**Expected:**
|
||||
- Database file exists (typically 100KB - 10MB+)
|
||||
- Contains observations and sessions
|
||||
- Recent observations visible
|
||||
|
||||
**If database missing or empty:**
|
||||
- New installation - this is normal, database will populate as you work
|
||||
- After `/clear` - sessions are marked complete but not deleted, data should persist
|
||||
- Corrupted database - backup and recreate:
|
||||
```bash
|
||||
cp ~/.claude-mem/claude-mem.db ~/.claude-mem/claude-mem.db.backup
|
||||
# Worker will recreate on next observation
|
||||
```
|
||||
|
||||
### 4. Check Dependencies Installation
|
||||
|
||||
Verify all required npm packages are installed:
|
||||
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
|
||||
# Check for critical packages
|
||||
ls node_modules/@anthropic-ai/claude-agent-sdk 2>&1 | head -1
|
||||
ls node_modules/better-sqlite3 2>&1 | head -1
|
||||
ls node_modules/express 2>&1 | head -1
|
||||
ls node_modules/pm2 2>&1 | head -1
|
||||
```
|
||||
|
||||
**Expected:** All critical packages present
|
||||
|
||||
**If dependencies missing:**
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
npm install
|
||||
```
|
||||
|
||||
### 5. Check Worker Logs
|
||||
|
||||
Review recent worker logs for errors:
|
||||
|
||||
```bash
|
||||
# View last 50 lines of worker logs
|
||||
pm2 logs claude-mem-worker --lines 50 --nostream
|
||||
|
||||
# Or use local pm2:
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
node_modules/.bin/pm2 logs claude-mem-worker --lines 50 --nostream
|
||||
|
||||
# Check for specific errors
|
||||
pm2 logs claude-mem-worker --lines 100 --nostream | grep -i "error\|exception\|failed"
|
||||
```
|
||||
|
||||
### 6. Test Viewer UI
|
||||
|
||||
Check if the web viewer is accessible:
|
||||
|
||||
```bash
|
||||
# Test viewer endpoint
|
||||
curl -s http://127.0.0.1:37777/ | head -20
|
||||
|
||||
# Test stats endpoint
|
||||
curl -s http://127.0.0.1:37777/api/stats
|
||||
```
|
||||
|
||||
**Expected:**
|
||||
- `/` returns HTML page with React viewer
|
||||
- `/api/stats` returns JSON with database counts
|
||||
|
||||
### 7. Check Port Configuration
|
||||
|
||||
Verify port settings and availability:
|
||||
|
||||
```bash
|
||||
# Check if custom port is configured
|
||||
cat ~/.claude-mem/settings.json 2>/dev/null
|
||||
cat ~/.claude/settings.json 2>/dev/null
|
||||
|
||||
# Check what's listening on default port
|
||||
lsof -i :37777 2>&1 || netstat -tlnp 2>&1 | grep 37777
|
||||
|
||||
# Test connectivity
|
||||
nc -zv 127.0.0.1 37777 2>&1
|
||||
```
|
||||
|
||||
## Full System Diagnosis Script
|
||||
|
||||
Run this comprehensive diagnostic script to collect all information:
|
||||
|
||||
```bash
|
||||
#!/bin/bash
|
||||
echo "=== Claude-Mem Troubleshooting Report ==="
|
||||
echo ""
|
||||
echo "1. Environment"
|
||||
echo " OS: $(uname -s)"
|
||||
echo ""
|
||||
echo "2. Plugin Installation"
|
||||
echo " Plugin directory exists: $([ -d ~/.claude/plugins/marketplaces/thedotmack ] && echo 'YES' || echo 'NO')"
|
||||
echo " Package version: $(grep '"version"' ~/.claude/plugins/marketplaces/thedotmack/package.json 2>/dev/null | head -1)"
|
||||
echo ""
|
||||
echo "3. Database"
|
||||
echo " Database exists: $([ -f ~/.claude-mem/claude-mem.db ] && echo 'YES' || echo 'NO')"
|
||||
echo " Database size: $(du -h ~/.claude-mem/claude-mem.db 2>/dev/null | cut -f1)"
|
||||
echo " Observation count: $(sqlite3 ~/.claude-mem/claude-mem.db 'SELECT COUNT(*) FROM observations;' 2>/dev/null || echo 'N/A')"
|
||||
echo " Session count: $(sqlite3 ~/.claude-mem/claude-mem.db 'SELECT COUNT(*) FROM sessions;' 2>/dev/null || echo 'N/A')"
|
||||
echo ""
|
||||
echo "4. Worker Service"
|
||||
PM2_PATH=$(which pm2 2>/dev/null || echo "~/.claude/plugins/marketplaces/thedotmack/node_modules/.bin/pm2")
|
||||
echo " PM2 path: $PM2_PATH"
|
||||
WORKER_STATUS=$($PM2_PATH jlist 2>/dev/null | grep -o '"name":"claude-mem-worker".*"status":"[^"]*"' | grep -o 'status":"[^"]*"' | cut -d'"' -f3 || echo 'not running')
|
||||
echo " Worker status: $WORKER_STATUS"
|
||||
echo " Health check: $(curl -s http://127.0.0.1:37777/health 2>/dev/null || echo 'FAILED')"
|
||||
echo ""
|
||||
echo "5. Configuration"
|
||||
echo " Port setting: $(cat ~/.claude-mem/settings.json 2>/dev/null | grep CLAUDE_MEM_WORKER_PORT || echo 'default (37777)')"
|
||||
echo " Observation count: $(cat ~/.claude/settings.json 2>/dev/null | grep CLAUDE_MEM_CONTEXT_OBSERVATIONS || echo 'default (50)')"
|
||||
echo ""
|
||||
echo "6. Recent Activity"
|
||||
echo " Latest observation: $(sqlite3 ~/.claude-mem/claude-mem.db 'SELECT created_at FROM observations ORDER BY created_at DESC LIMIT 1;' 2>/dev/null || echo 'N/A')"
|
||||
echo " Latest session: $(sqlite3 ~/.claude-mem/claude-mem.db 'SELECT created_at FROM sessions ORDER BY created_at DESC LIMIT 1;' 2>/dev/null || echo 'N/A')"
|
||||
echo ""
|
||||
echo "=== End Report ==="
|
||||
```
|
||||
|
||||
Save this as `/tmp/claude-mem-diagnostics.sh` and run:
|
||||
```bash
|
||||
bash /tmp/claude-mem-diagnostics.sh
|
||||
```
|
||||
|
||||
## Reporting Issues
|
||||
|
||||
If troubleshooting doesn't resolve the issue, collect this information for a bug report:
|
||||
|
||||
1. Full diagnostic report (run script above)
|
||||
2. Worker logs: `pm2 logs claude-mem-worker --lines 100 --nostream`
|
||||
3. Your setup:
|
||||
- Claude version: Check with Claude
|
||||
- OS: `uname -a`
|
||||
- Node version: `node --version`
|
||||
- Plugin version: In package.json
|
||||
4. Steps to reproduce the issue
|
||||
5. Expected vs actual behavior
|
||||
|
||||
Post to: https://github.com/thedotmack/claude-mem/issues
|
||||
190
skills/troubleshoot/operations/reference.md
Normal file
190
skills/troubleshoot/operations/reference.md
Normal file
@@ -0,0 +1,190 @@
|
||||
# Quick Commands Reference
|
||||
|
||||
Essential commands for troubleshooting claude-mem.
|
||||
|
||||
## Worker Management
|
||||
|
||||
```bash
|
||||
# Check worker status
|
||||
pm2 status | grep claude-mem-worker
|
||||
pm2 jlist | grep claude-mem-worker # JSON format
|
||||
|
||||
# Start worker
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
pm2 start ecosystem.config.cjs
|
||||
|
||||
# Restart worker
|
||||
pm2 restart claude-mem-worker
|
||||
|
||||
# Stop worker
|
||||
pm2 stop claude-mem-worker
|
||||
|
||||
# Delete worker (for clean restart)
|
||||
pm2 delete claude-mem-worker
|
||||
|
||||
# View logs
|
||||
pm2 logs claude-mem-worker
|
||||
|
||||
# View last N lines
|
||||
pm2 logs claude-mem-worker --lines 50 --nostream
|
||||
|
||||
# Clear logs
|
||||
pm2 flush claude-mem-worker
|
||||
```
|
||||
|
||||
## Health Checks
|
||||
|
||||
```bash
|
||||
# Check worker health (default port)
|
||||
curl -s http://127.0.0.1:37777/health
|
||||
|
||||
# Check viewer stats
|
||||
curl -s http://127.0.0.1:37777/api/stats
|
||||
|
||||
# Open viewer in browser
|
||||
open http://127.0.0.1:37777
|
||||
|
||||
# Test custom port
|
||||
PORT=37778
|
||||
curl -s http://127.0.0.1:$PORT/health
|
||||
```
|
||||
|
||||
## Database Queries
|
||||
|
||||
```bash
|
||||
# Observation count
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM observations;"
|
||||
|
||||
# Session count
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT COUNT(*) FROM sessions;"
|
||||
|
||||
# Recent observations
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT created_at, type, title FROM observations ORDER BY created_at DESC LIMIT 10;"
|
||||
|
||||
# Recent sessions
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT created_at, request FROM sessions ORDER BY created_at DESC LIMIT 5;"
|
||||
|
||||
# Database size
|
||||
du -h ~/.claude-mem/claude-mem.db
|
||||
|
||||
# Database integrity check
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "PRAGMA integrity_check;"
|
||||
|
||||
# Projects in database
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "SELECT DISTINCT project FROM observations ORDER BY project;"
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
```bash
|
||||
# View current settings
|
||||
cat ~/.claude-mem/settings.json
|
||||
cat ~/.claude/settings.json
|
||||
|
||||
# Change worker port
|
||||
echo '{"env":{"CLAUDE_MEM_WORKER_PORT":"37778"}}' > ~/.claude-mem/settings.json
|
||||
|
||||
# Change context observation count
|
||||
# Edit ~/.claude/settings.json and add:
|
||||
{
|
||||
"env": {
|
||||
"CLAUDE_MEM_CONTEXT_OBSERVATIONS": "25"
|
||||
}
|
||||
}
|
||||
|
||||
# Change AI model
|
||||
{
|
||||
"env": {
|
||||
"CLAUDE_MEM_MODEL": "claude-haiku-4-5"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Plugin Management
|
||||
|
||||
```bash
|
||||
# Navigate to plugin directory
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
|
||||
# Check plugin version
|
||||
grep '"version"' package.json
|
||||
|
||||
# Reinstall dependencies
|
||||
npm install
|
||||
|
||||
# View package.json
|
||||
cat package.json
|
||||
```
|
||||
|
||||
## Port Diagnostics
|
||||
|
||||
```bash
|
||||
# Check what's using port 37777
|
||||
lsof -i :37777
|
||||
netstat -tlnp | grep 37777
|
||||
|
||||
# Test port connectivity
|
||||
nc -zv 127.0.0.1 37777
|
||||
curl -v http://127.0.0.1:37777/health
|
||||
```
|
||||
|
||||
## Log Analysis
|
||||
|
||||
```bash
|
||||
# Search logs for errors
|
||||
pm2 logs claude-mem-worker --lines 100 --nostream | grep -i "error"
|
||||
|
||||
# Search for specific keyword
|
||||
pm2 logs claude-mem-worker --lines 100 --nostream | grep "keyword"
|
||||
|
||||
# Follow logs in real-time
|
||||
pm2 logs claude-mem-worker
|
||||
|
||||
# Show only error logs
|
||||
pm2 logs claude-mem-worker --err
|
||||
```
|
||||
|
||||
## File Locations
|
||||
|
||||
```bash
|
||||
# Plugin directory
|
||||
~/.claude/plugins/marketplaces/thedotmack/
|
||||
|
||||
# Database
|
||||
~/.claude-mem/claude-mem.db
|
||||
|
||||
# Settings
|
||||
~/.claude-mem/settings.json
|
||||
~/.claude/settings.json
|
||||
|
||||
# Chroma vector database
|
||||
~/.claude-mem/chroma/
|
||||
|
||||
# Usage logs
|
||||
~/.claude-mem/usage-logs/
|
||||
|
||||
# PM2 logs
|
||||
~/.pm2/logs/
|
||||
```
|
||||
|
||||
## System Information
|
||||
|
||||
```bash
|
||||
# OS version
|
||||
uname -a
|
||||
|
||||
# Node version
|
||||
node --version
|
||||
|
||||
# NPM version
|
||||
npm --version
|
||||
|
||||
# PM2 version
|
||||
pm2 --version
|
||||
|
||||
# SQLite version
|
||||
sqlite3 --version
|
||||
|
||||
# Check disk space
|
||||
df -h ~/.claude-mem/
|
||||
```
|
||||
308
skills/troubleshoot/operations/worker.md
Normal file
308
skills/troubleshoot/operations/worker.md
Normal file
@@ -0,0 +1,308 @@
|
||||
# Worker Service Diagnostics
|
||||
|
||||
PM2 worker-specific troubleshooting for claude-mem.
|
||||
|
||||
## PM2 Worker Overview
|
||||
|
||||
The claude-mem worker is a persistent background service managed by PM2. It:
|
||||
- Runs Express.js server on port 37777 (default)
|
||||
- Processes observations asynchronously
|
||||
- Serves the viewer UI
|
||||
- Provides search API endpoints
|
||||
|
||||
## Check Worker Status
|
||||
|
||||
### Basic Status Check
|
||||
|
||||
```bash
|
||||
# List all PM2 processes
|
||||
pm2 list
|
||||
|
||||
# JSON format (parseable)
|
||||
pm2 jlist
|
||||
|
||||
# Filter for claude-mem-worker
|
||||
pm2 status | grep claude-mem-worker
|
||||
```
|
||||
|
||||
**Expected output:**
|
||||
```
|
||||
│ claude-mem-worker │ online │ 12345 │ 0 │ 45m │ 0% │ 85.6mb │
|
||||
```
|
||||
|
||||
**Status meanings:**
|
||||
- `online` - Worker running correctly
|
||||
- `stopped` - Worker stopped (normal shutdown)
|
||||
- `errored` - Worker crashed (check logs)
|
||||
- `stopping` - Worker shutting down
|
||||
- Not listed - Worker never started
|
||||
|
||||
### Detailed Worker Info
|
||||
|
||||
```bash
|
||||
# Show detailed information
|
||||
pm2 show claude-mem-worker
|
||||
|
||||
# JSON format
|
||||
pm2 jlist | grep -A 20 '"name":"claude-mem-worker"'
|
||||
```
|
||||
|
||||
## Worker Health Endpoint
|
||||
|
||||
The worker exposes a health endpoint at `/health`:
|
||||
|
||||
```bash
|
||||
# Check health (default port)
|
||||
curl -s http://127.0.0.1:37777/health
|
||||
|
||||
# With custom port
|
||||
PORT=$(grep CLAUDE_MEM_WORKER_PORT ~/.claude-mem/settings.json | grep -o '[0-9]\+' || echo "37777")
|
||||
curl -s http://127.0.0.1:$PORT/health
|
||||
```
|
||||
|
||||
**Expected response:** `{"status":"ok"}`
|
||||
|
||||
**Error responses:**
|
||||
- Connection refused - Worker not running
|
||||
- Timeout - Worker hung (restart needed)
|
||||
- Empty response - Worker crashed mid-request
|
||||
|
||||
## Worker Logs
|
||||
|
||||
### View Recent Logs
|
||||
|
||||
```bash
|
||||
# Last 50 lines
|
||||
pm2 logs claude-mem-worker --lines 50 --nostream
|
||||
|
||||
# Last 200 lines
|
||||
pm2 logs claude-mem-worker --lines 200 --nostream
|
||||
|
||||
# Follow logs in real-time
|
||||
pm2 logs claude-mem-worker
|
||||
```
|
||||
|
||||
### Search Logs for Errors
|
||||
|
||||
```bash
|
||||
# Find errors
|
||||
pm2 logs claude-mem-worker --lines 500 --nostream | grep -i "error"
|
||||
|
||||
# Find exceptions
|
||||
pm2 logs claude-mem-worker --lines 500 --nostream | grep -i "exception"
|
||||
|
||||
# Find failed requests
|
||||
pm2 logs claude-mem-worker --lines 500 --nostream | grep -i "failed"
|
||||
|
||||
# All error patterns
|
||||
pm2 logs claude-mem-worker --lines 500 --nostream | grep -iE "error|exception|failed|crash"
|
||||
```
|
||||
|
||||
### Common Log Patterns
|
||||
|
||||
**Good startup:**
|
||||
```
|
||||
Worker service started on port 37777
|
||||
Database initialized
|
||||
Express server listening
|
||||
```
|
||||
|
||||
**Database errors:**
|
||||
```
|
||||
Error: SQLITE_ERROR
|
||||
Error initializing database
|
||||
Database locked
|
||||
```
|
||||
|
||||
**Port conflicts:**
|
||||
```
|
||||
Error: listen EADDRINUSE
|
||||
Port 37777 already in use
|
||||
```
|
||||
|
||||
**Crashes:**
|
||||
```
|
||||
PM2 | App [claude-mem-worker] exited with code [1]
|
||||
PM2 | App [claude-mem-worker] will restart in 100ms
|
||||
```
|
||||
|
||||
## Starting the Worker
|
||||
|
||||
### Basic Start
|
||||
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
pm2 start ecosystem.config.cjs
|
||||
```
|
||||
|
||||
### Start with Local PM2
|
||||
|
||||
If `pm2` command not in PATH:
|
||||
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
node_modules/.bin/pm2 start ecosystem.config.cjs
|
||||
```
|
||||
|
||||
### Force Restart
|
||||
|
||||
```bash
|
||||
# Restart if already running
|
||||
pm2 restart claude-mem-worker
|
||||
|
||||
# Delete and start fresh
|
||||
pm2 delete claude-mem-worker
|
||||
pm2 start ecosystem.config.cjs
|
||||
```
|
||||
|
||||
## Stopping the Worker
|
||||
|
||||
```bash
|
||||
# Graceful stop
|
||||
pm2 stop claude-mem-worker
|
||||
|
||||
# Delete completely (also removes from PM2 list)
|
||||
pm2 delete claude-mem-worker
|
||||
```
|
||||
|
||||
## Worker Not Starting
|
||||
|
||||
### Diagnostic Steps
|
||||
|
||||
1. **Try manual start to see error:**
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
node plugin/scripts/worker-service.cjs
|
||||
```
|
||||
This runs the worker directly without PM2, showing full error output.
|
||||
|
||||
2. **Check PM2 itself:**
|
||||
```bash
|
||||
which pm2
|
||||
pm2 --version
|
||||
```
|
||||
If PM2 not found, dependencies not installed.
|
||||
|
||||
3. **Check dependencies:**
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
ls node_modules/@anthropic-ai/claude-agent-sdk
|
||||
ls node_modules/better-sqlite3
|
||||
ls node_modules/express
|
||||
ls node_modules/pm2
|
||||
```
|
||||
|
||||
4. **Check port availability:**
|
||||
```bash
|
||||
lsof -i :37777
|
||||
```
|
||||
If port in use, either kill that process or change claude-mem port.
|
||||
|
||||
### Common Fixes
|
||||
|
||||
**Dependencies missing:**
|
||||
```bash
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
npm install
|
||||
pm2 start ecosystem.config.cjs
|
||||
```
|
||||
|
||||
**Port conflict:**
|
||||
```bash
|
||||
echo '{"env":{"CLAUDE_MEM_WORKER_PORT":"37778"}}' > ~/.claude-mem/settings.json
|
||||
pm2 restart claude-mem-worker
|
||||
```
|
||||
|
||||
**Corrupted PM2:**
|
||||
```bash
|
||||
pm2 kill # Stop PM2 daemon
|
||||
cd ~/.claude/plugins/marketplaces/thedotmack/
|
||||
pm2 start ecosystem.config.cjs
|
||||
```
|
||||
|
||||
## Worker Crashing Repeatedly
|
||||
|
||||
If worker keeps restarting (check with `pm2 status` showing high restart count):
|
||||
|
||||
### Find the Cause
|
||||
|
||||
1. **Check error logs:**
|
||||
```bash
|
||||
pm2 logs claude-mem-worker --err --lines 100 --nostream
|
||||
```
|
||||
|
||||
2. **Look for crash pattern:**
|
||||
```bash
|
||||
pm2 logs claude-mem-worker --lines 200 --nostream | grep -A 5 "exited with code"
|
||||
```
|
||||
|
||||
### Common Crash Causes
|
||||
|
||||
**Database corruption:**
|
||||
```bash
|
||||
sqlite3 ~/.claude-mem/claude-mem.db "PRAGMA integrity_check;"
|
||||
```
|
||||
If fails, backup and recreate database.
|
||||
|
||||
**Out of memory:**
|
||||
Check if database is too large or memory leak. Restart:
|
||||
```bash
|
||||
pm2 restart claude-mem-worker
|
||||
```
|
||||
|
||||
**Port conflict race condition:**
|
||||
Another process grabbing port intermittently. Change port:
|
||||
```bash
|
||||
echo '{"env":{"CLAUDE_MEM_WORKER_PORT":"37778"}}' > ~/.claude-mem/settings.json
|
||||
pm2 restart claude-mem-worker
|
||||
```
|
||||
|
||||
## PM2 Management Commands
|
||||
|
||||
```bash
|
||||
# List processes
|
||||
pm2 list
|
||||
pm2 jlist # JSON format
|
||||
|
||||
# Show detailed info
|
||||
pm2 show claude-mem-worker
|
||||
|
||||
# Monitor resources
|
||||
pm2 monit
|
||||
|
||||
# Clear logs
|
||||
pm2 flush claude-mem-worker
|
||||
|
||||
# Restart PM2 daemon
|
||||
pm2 kill
|
||||
pm2 resurrect # Restore saved processes
|
||||
|
||||
# Save current process list
|
||||
pm2 save
|
||||
|
||||
# Update PM2
|
||||
npm install -g pm2
|
||||
```
|
||||
|
||||
## Testing Worker Endpoints
|
||||
|
||||
Once worker is running, test all endpoints:
|
||||
|
||||
```bash
|
||||
# Health check
|
||||
curl -s http://127.0.0.1:37777/health
|
||||
|
||||
# Viewer HTML
|
||||
curl -s http://127.0.0.1:37777/ | head -20
|
||||
|
||||
# Stats API
|
||||
curl -s http://127.0.0.1:37777/api/stats
|
||||
|
||||
# Search API
|
||||
curl -s "http://127.0.0.1:37777/api/search/observations?q=test&format=index"
|
||||
|
||||
# Prompts API
|
||||
curl -s "http://127.0.0.1:37777/api/prompts?limit=5"
|
||||
```
|
||||
|
||||
All should return appropriate responses (HTML for viewer, JSON for APIs).
|
||||
Reference in New Issue
Block a user