Initial commit

This commit is contained in:
Zhongwei Li
2025-11-30 09:01:40 +08:00
commit 7ad6af0a78
28 changed files with 4312 additions and 0 deletions

123
skills/mem-search/SKILL.md Normal file
View File

@@ -0,0 +1,123 @@
---
name: mem-search
description: Search claude-mem's persistent cross-session memory database. Use when user asks "did we already solve this?", "how did we do X last time?", or needs work from previous sessions.
---
# Memory Search
Search past work across all sessions. Simple workflow: search → get IDs → fetch details by ID.
## When to Use
Use when users ask about PREVIOUS sessions (not current conversation):
- "Did we already fix this?"
- "How did we solve X last time?"
- "What happened last week?"
## The Workflow
**ALWAYS follow this exact flow:**
1. **Search** - Get an index of results with IDs
2. **Review** - Look at titles/dates, pick relevant IDs
3. **Fetch** - Get full details ONLY for those IDs
### Step 1: Search Everything
```bash
curl "http://localhost:37777/api/search?query=authentication&format=index&limit=5"
```
**Required parameters:**
- `query` - Search term
- `format=index` - ALWAYS start with index (lightweight)
- `limit=5` - Start small (3-5 results)
**Returns:**
```
1. [feature] Added JWT authentication
Date: 11/17/2025, 3:48:45 PM
ID: 11131
2. [bugfix] Fixed auth token expiration
Date: 11/16/2025, 2:15:22 PM
ID: 10942
```
### Step 2: Pick IDs
Review the index results. Identify which IDs are actually relevant. Discard the rest.
### Step 3: Fetch by ID
For each relevant ID, fetch full details:
```bash
# Fetch observation
curl "http://localhost:37777/api/observation/11131"
# Fetch session
curl "http://localhost:37777/api/session/2005"
# Fetch prompt
curl "http://localhost:37777/api/prompt/5421"
```
**ID formats:**
- Observations: Just the number (11131)
- Sessions: Just the number (2005) from "S2005"
- Prompts: Just the number (5421)
## Search Parameters
**Basic:**
- `query` - What to search for (required)
- `format` - "index" or "full" (always use "index" first)
- `limit` - How many results (default 5, max 100)
**Filters (optional):**
- `type` - Filter to "observations", "sessions", or "prompts"
- `project` - Filter by project name
- `dateRange[start]` - Start date (YYYY-MM-DD)
- `dateRange[end]` - End date (YYYY-MM-DD)
- `obs_type` - Filter observations by: bugfix, feature, decision, discovery, change
## Examples
**Find recent bug fixes:**
```bash
curl "http://localhost:37777/api/search?query=bug&type=observations&obs_type=bugfix&format=index&limit=5"
```
**Find what happened last week:**
```bash
curl "http://localhost:37777/api/search?query=&type=observations&dateRange[start]=2025-11-11&format=index&limit=10"
```
**Search everything:**
```bash
curl "http://localhost:37777/api/search?query=database+migration&format=index&limit=5"
```
## Why This Workflow?
**Token efficiency:**
- Index format: ~50-100 tokens per result
- Full format: ~500-1000 tokens per result
- **10x difference** - only fetch full when you know it's relevant
**Clarity:**
- See everything first
- Pick what matters
- Get details only for what you need
## Error Handling
If search fails, tell the user the worker isn't available and suggest:
```bash
pm2 list # Check if worker is running
```
---
**Remember:** ALWAYS search with format=index first. ALWAYS fetch by ID for details. The IDs are there for a reason - USE THEM.

View File

View File

@@ -0,0 +1,124 @@
# Search by Concept
Find observations tagged with specific concepts.
## When to Use
- User asks: "What discoveries did we make?"
- User asks: "What patterns did we identify?"
- User asks: "What gotchas did we encounter?"
- Looking for observations with semantic tags
## Command
```bash
curl -s "http://localhost:37777/api/search/by-concept?concept=discovery&format=index&limit=5"
```
## Parameters
- **concept** (required): Concept tag to search for
- `discovery` - New discoveries and insights
- `problem-solution` - Problems and their solutions
- `what-changed` - Change descriptions
- `how-it-works` - Explanations of mechanisms
- `pattern` - Identified patterns
- `gotcha` - Edge cases and gotchas
- `change` - General changes
- **format**: "index" (summary) or "full" (complete details). Default: "full"
- **limit**: Number of results (default: 20, max: 100)
- **project**: Filter by project name (optional)
- **dateRange**: Filter by date range (optional)
## When to Use Each Format
**Use format=index for:**
- Quick overviews of observations by concept
- Finding IDs for deeper investigation
- Listing multiple results
- **Token cost: ~50-100 per result**
**Use format=full for:**
- Complete details including narrative, facts, files, concepts
- Understanding the full context of specific observations
- **Token cost: ~500-1000 per result**
## Example Response (format=index)
```json
{
"concept": "discovery",
"count": 3,
"format": "index",
"results": [
{
"id": 1240,
"type": "discovery",
"title": "Worker service uses PM2 for process management",
"subtitle": "Discovered persistent background worker pattern",
"created_at_epoch": 1699564800000,
"project": "claude-mem",
"concepts": ["discovery", "how-it-works"]
}
]
}
```
## How to Present Results
For format=index, present as a compact list:
```markdown
Found 3 observations tagged with "discovery":
🔵 **#1240** Worker service uses PM2 for process management
> Discovered persistent background worker pattern
> Nov 9, 2024 • claude-mem
> Tags: discovery, how-it-works
🔵 **#1241** FTS5 full-text search enables instant searches
> SQLite FTS5 virtual tables provide sub-100ms search
> Nov 9, 2024 • claude-mem
> Tags: discovery, pattern
```
For complete formatting guidelines, see [formatting.md](formatting.md).
## Available Concepts
| Concept | Description | When to Use |
|---------|-------------|-------------|
| `discovery` | New discoveries and insights | Finding what was learned |
| `problem-solution` | Problems and their solutions | Finding how issues were resolved |
| `what-changed` | Change descriptions | Understanding what changed |
| `how-it-works` | Explanations of mechanisms | Learning how things work |
| `pattern` | Identified patterns | Finding design patterns |
| `gotcha` | Edge cases and gotchas | Learning about pitfalls |
| `change` | General changes | Tracking modifications |
## Error Handling
**Missing concept parameter:**
```json
{"error": "Missing required parameter: concept"}
```
Fix: Add the concept parameter
**Invalid concept:**
```json
{"error": "Invalid concept: foobar. Valid concepts: discovery, problem-solution, what-changed, how-it-works, pattern, gotcha, change"}
```
Fix: Use one of the valid concept values
## Tips
1. Use format=index first to see overview
2. Start with limit=5-10 to avoid token overload
3. Combine concepts with type filtering for precision
4. Use `discovery` for learning what was found during investigation
5. Use `problem-solution` for finding how past issues were resolved
**Token Efficiency:**
- Start with format=index (~50-100 tokens per result)
- Use format=full only for relevant items (~500-1000 tokens per result)
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)

View File

@@ -0,0 +1,127 @@
# Search by File
Find all work related to a specific file path.
## When to Use
- User asks: "What changes to auth/login.ts?"
- User asks: "What work was done on this file?"
- User asks: "Show me the history of src/services/worker.ts"
- Looking for all observations that reference a file
## Command
```bash
curl -s "http://localhost:37777/api/search/by-file?filePath=src/services/worker-service.ts&format=index&limit=10"
```
## Parameters
- **filePath** (required): File path to search for (supports partial matching)
- Full path: `src/services/worker-service.ts`
- Partial path: `worker-service.ts`
- Directory: `src/hooks/`
- **format**: "index" (summary) or "full" (complete details). Default: "full"
- **limit**: Number of results (default: 20, max: 100)
- **project**: Filter by project name (optional)
- **dateRange**: Filter by date range (optional)
## When to Use Each Format
**Use format=index for:**
- Quick overviews of work on a file
- Finding IDs for deeper investigation
- Listing multiple changes
- **Token cost: ~50-100 per result**
**Use format=full for:**
- Complete details including narrative, facts, files, concepts
- Understanding the full context of specific changes
- **Token cost: ~500-1000 per result**
## Example Response (format=index)
```json
{
"filePath": "src/services/worker-service.ts",
"count": 8,
"format": "index",
"results": [
{
"id": 1245,
"type": "refactor",
"title": "Simplified worker health check logic",
"subtitle": "Removed redundant PM2 status check",
"created_at_epoch": 1699564800000,
"project": "claude-mem",
"files": ["src/services/worker-service.ts", "src/services/worker-utils.ts"]
}
]
}
```
## How to Present Results
For format=index, present as a compact list:
```markdown
Found 8 observations related to "src/services/worker-service.ts":
🔄 **#1245** Simplified worker health check logic
> Removed redundant PM2 status check
> Nov 9, 2024 • claude-mem
> Files: worker-service.ts, worker-utils.ts
🟣 **#1246** Added SSE endpoint for real-time updates
> Implemented Server-Sent Events for viewer UI
> Nov 8, 2024 • claude-mem
> Files: worker-service.ts
```
For complete formatting guidelines, see [formatting.md](formatting.md).
## Partial Path Matching
The file path parameter supports partial matching:
```bash
# These all match "src/services/worker-service.ts"
curl -s "http://localhost:37777/api/search/by-file?filePath=worker-service.ts&format=index"
curl -s "http://localhost:37777/api/search/by-file?filePath=services/worker&format=index"
curl -s "http://localhost:37777/api/search/by-file?filePath=worker-service&format=index"
```
## Directory Searches
Search for all work in a directory:
```bash
curl -s "http://localhost:37777/api/search/by-file?filePath=src/hooks/&format=index&limit=20"
```
## Error Handling
**Missing filePath parameter:**
```json
{"error": "Missing required parameter: filePath"}
```
Fix: Add the filePath parameter
**No results found:**
```json
{"filePath": "nonexistent.ts", "count": 0, "results": []}
```
Response: "No observations found for 'nonexistent.ts'. Try a partial path or check the spelling."
## Tips
1. Use format=index first to see overview of all changes
2. Start with partial paths (e.g., filename only) for broader matches
3. Use full paths when you need specific file matches
4. Combine with dateRange to see recent changes: `?filePath=worker.ts&dateRange[start]=2024-11-01`
5. Use directory searches to see all work in a module
**Token Efficiency:**
- Start with format=index (~50-100 tokens per result)
- Use format=full only for relevant items (~500-1000 tokens per result)
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)

View File

@@ -0,0 +1,123 @@
# Search by Type
Find observations by type: bugfix, feature, refactor, decision, discovery, or change.
## When to Use
- User asks: "What bugs did we fix?"
- User asks: "What features did we add?"
- User asks: "What decisions did we make?"
- Looking for specific types of work
## Command
```bash
curl -s "http://localhost:37777/api/search/by-type?type=bugfix&format=index&limit=5"
```
## Parameters
- **type** (required): One or more types (comma-separated)
- `bugfix` - Bug fixes
- `feature` - New features
- `refactor` - Code refactoring
- `decision` - Architectural/design decisions
- `discovery` - Discoveries and insights
- `change` - General changes
- **format**: "index" (summary) or "full" (complete details). Default: "full"
- **limit**: Number of results (default: 20, max: 100)
- **project**: Filter by project name (optional)
- **dateRange**: Filter by date range (optional)
## When to Use Each Format
**Use format=index for:**
- Quick overviews of work by type
- Finding IDs for deeper investigation
- Listing multiple results
- **Token cost: ~50-100 per result**
**Use format=full for:**
- Complete details including narrative, facts, files, concepts
- Understanding the full context of specific observations
- **Token cost: ~500-1000 per result**
## Example Response (format=index)
```json
{
"type": "bugfix",
"count": 5,
"format": "index",
"results": [
{
"id": 1235,
"type": "bugfix",
"title": "Fixed token expiration edge case",
"subtitle": "Handled race condition in refresh flow",
"created_at_epoch": 1699564800000,
"project": "api-server"
}
]
}
```
## How to Present Results
For format=index, present as a compact list with type emojis:
```markdown
Found 5 bugfixes:
🔴 **#1235** Fixed token expiration edge case
> Handled race condition in refresh flow
> Nov 9, 2024 • api-server
🔴 **#1236** Resolved memory leak in worker
> Fixed event listener cleanup
> Nov 8, 2024 • worker-service
```
**Type Emojis:**
- 🔴 bugfix
- 🟣 feature
- 🔄 refactor
- 🔵 discovery
- 🧠 decision
- ✅ change
For complete formatting guidelines, see [formatting.md](formatting.md).
## Multiple Types
To search for multiple types:
```bash
curl -s "http://localhost:37777/api/search/by-type?type=bugfix,feature&format=index&limit=10"
```
## Error Handling
**Missing type parameter:**
```json
{"error": "Missing required parameter: type"}
```
Fix: Add the type parameter
**Invalid type:**
```json
{"error": "Invalid type: foobar. Valid types: bugfix, feature, refactor, decision, discovery, change"}
```
Fix: Use one of the valid type values
## Tips
1. Use format=index first to see overview
2. Start with limit=5-10 to avoid token overload
3. Combine with dateRange for recent work: `?type=bugfix&dateRange[start]=2024-11-01`
4. Use project filtering when working on one codebase
**Token Efficiency:**
- Start with format=index (~50-100 tokens per result)
- Use format=full only for relevant items (~500-1000 tokens per result)
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)

View File

@@ -0,0 +1,251 @@
# Common Workflows
Step-by-step guides for typical user requests using the search API.
## Workflow 1: Understanding Past Work
**User asks:** "What did we do last session?" or "Catch me up on recent work"
**Steps:**
1. **Get recent context** (fastest path):
```bash
curl -s "http://localhost:37777/api/context/recent?limit=3"
```
2. **Present as narrative:**
```markdown
## Recent Work
### Session #545 - Nov 9, 2024
Implemented JWT authentication system
**Completed:**
- Added token-based auth with refresh tokens
- Created JWT signing and verification logic
**Key Learning:** JWT expiration requires careful handling of refresh race conditions
```
**Why this workflow:**
- Single request gets both sessions and observations
- Optimized for "catch me up" questions
- ~1,500-2,500 tokens for 3 sessions
---
## Workflow 2: Finding Specific Bug Fixes
**User asks:** "What bugs did we fix?" or "Show me recent bug fixes"
**Steps:**
1. **Search by type** (index format first):
```bash
curl -s "http://localhost:37777/api/search/by-type?type=bugfix&format=index&limit=5"
```
2. **Review index results**, identify relevant items
3. **Get full details** for specific bugs:
```bash
curl -s "http://localhost:37777/api/search/by-type?type=bugfix&format=full&limit=1&offset=2"
```
4. **Present findings:**
```markdown
Found 5 bug fixes:
🔴 **#1235** Fixed token expiration edge case
> Handled race condition in refresh flow
> Nov 9, 2024 • api-server
[Click for full details on #1235]
```
**Why this workflow:**
- Progressive disclosure: index first, full details selectively
- Type-specific search is more efficient than generic search
- ~250-500 tokens for index, ~750-1000 per full detail
---
## Workflow 3: Understanding File History
**User asks:** "What changes to auth/login.ts?" or "Show me work on this file"
**Steps:**
1. **Search by file** (index format):
```bash
curl -s "http://localhost:37777/api/search/by-file?filePath=auth/login.ts&format=index&limit=10"
```
2. **Review chronological changes**
3. **Get full details** for specific changes:
```bash
curl -s "http://localhost:37777/api/search/by-file?filePath=auth/login.ts&format=full&limit=1&offset=3"
```
4. **Present as file timeline:**
```markdown
## History of auth/login.ts
🟣 **#1230** Added JWT authentication (Nov 9)
🔴 **#1235** Fixed token expiration bug (Nov 9)
🔄 **#1240** Refactored auth flow (Nov 8)
```
**Why this workflow:**
- File-specific search finds all related work
- Index format shows chronological overview
- Selective full details for deep dives
---
## Workflow 4: Timeline Investigation
**User asks:** "What was happening when we deployed?" or "Show me context around that bug fix"
**Steps:**
1. **Find the event** using search:
```bash
curl -s "http://localhost:37777/api/search/observations?query=deployment&format=index&limit=5"
```
2. **Note observation ID** (e.g., #1234)
3. **Get timeline context**:
```bash
curl -s "http://localhost:37777/api/timeline/context?anchor=1234&depth_before=10&depth_after=10"
```
4. **Present as chronological narrative:**
```markdown
## Timeline: Deployment
### Before (10 records)
**2:45 PM** - 🟣 Prepared deployment scripts
**2:50 PM** - 💬 User asked: "Are we ready to deploy?"
### ⭐ Anchor Point (2:55 PM)
🎯 **Observation #1234**: Deployed to production
### After (10 records)
**3:00 PM** - 🔴 Fixed post-deployment routing issue
```
**Why this workflow:**
- Timeline shows temporal context (what happened before/after)
- Captures causality between events
- All record types (observations, sessions, prompts) interleaved
---
## Workflow 5: Quick Timeline (One Request)
**User asks:** "Timeline of authentication work"
**Steps:**
1. **Use timeline-by-query** (auto mode):
```bash
curl -s "http://localhost:37777/api/timeline/by-query?query=authentication&mode=auto&depth_before=10&depth_after=10"
```
2. **Present timeline directly:**
```markdown
## Timeline: Authentication
**Best Match:** 🟣 Observation #1234 - Implemented JWT authentication
### Context (21 records)
[... timeline around best match ...]
```
**Why this workflow:**
- Single request combines search + timeline
- Fastest path when query is specific
- Auto mode uses top result as anchor
**Alternative:** Use interactive mode for broad queries:
```bash
curl -s "http://localhost:37777/api/timeline/by-query?query=auth&mode=interactive&limit=5"
```
Then choose anchor manually.
---
## Workflow 6: Search Composition
**User asks:** "What features did we add to the authentication system recently?"
**Steps:**
1. **Combine filters** for precision:
```bash
curl -s "http://localhost:37777/api/search/observations?query=authentication&type=feature&dateRange[start]=2024-11-01&format=index&limit=10"
```
2. **Review filtered results**
3. **Get full details** for relevant features:
```bash
curl -s "http://localhost:37777/api/search/observations?query=authentication&type=feature&format=full&limit=1&offset=2"
```
4. **Present findings:**
```markdown
Found 10 authentication features added in November:
🟣 **#1234** Implemented JWT authentication (Nov 9)
🟣 **#1236** Added refresh token rotation (Nov 9)
🟣 **#1238** Implemented OAuth2 flow (Nov 7)
```
**Why this workflow:**
- Multiple filters narrow results before requesting full details
- Type + query + dateRange = precise targeting
- Progressive disclosure: index first, full details selectively
---
## Workflow Selection Guide
| User Request | Workflow | Operation | Token Cost |
|--------------|----------|-----------|------------|
| "What did we do last session?" | #1 | recent-context | 1,500-2,500 |
| "What bugs did we fix?" | #2 | by-type | 500-3,000 |
| "What changes to file.ts?" | #3 | by-file | 500-3,000 |
| "What was happening then?" | #4 | search → timeline | 3,500-6,000 |
| "Timeline of X work" | #5 | timeline-by-query | 3,000-4,000 |
| "Recent features added?" | #6 | observations + filters | 500-3,000 |
## General Principles
1. **Start with index format** - Always use `format=index` first
2. **Use specialized tools** - by-type, by-file, by-concept when applicable
3. **Compose operations** - Combine search + timeline for investigations
4. **Filter early** - Use type, dateRange, project to narrow before expanding
5. **Progressive disclosure** - Load full details only for relevant items
## Token Budget Awareness
**Quick queries** (500-1,500 tokens):
- Recent context (limit=3)
- Index search (limit=5-10)
- Filtered searches
**Medium queries** (1,500-4,000 tokens):
- Recent context (limit=5-10)
- Full details (3-5 items)
- Timeline (depth 10/10)
**Deep queries** (4,000-8,000 tokens):
- Timeline (depth 20/20)
- Full details (10+ items)
- Multiple composed operations
Always start with minimal token investment, expand only when needed.

View File

@@ -0,0 +1,403 @@
# Response Formatting Guidelines
How to present search results to users for maximum clarity and usefulness.
## General Principles
1. **Progressive disclosure** - Show index results first, full details on demand
2. **Visual hierarchy** - Use emojis, bold, and structure for scannability
3. **Context-aware** - Tailor presentation to user's question
4. **Actionable** - Include IDs for follow-up queries
5. **Token-efficient** - Balance detail with token budget
---
## Format: Index Results
**When to use:** First response to searches, overviews, multiple results
**Structure:**
```markdown
Found {count} results for "{query}":
{emoji} **#{id}** {title}
> {subtitle}
> {date} • {project}
```
**Example:**
```markdown
Found 5 results for "authentication":
🟣 **#1234** Implemented JWT authentication
> Added token-based auth with refresh tokens
> Nov 9, 2024 • api-server
🔴 **#1235** Fixed token expiration edge case
> Handled race condition in refresh flow
> Nov 9, 2024 • api-server
```
**Type Emojis:**
- 🔴 bugfix
- 🟣 feature
- 🔄 refactor
- 🔵 discovery
- 🧠 decision
- ✅ change
- 🎯 session
- 💬 prompt
**What to include:**
- ✅ ID (for follow-up)
- ✅ Type emoji
- ✅ Title
- ✅ Subtitle (if available)
- ✅ Date (human-readable)
- ✅ Project name
- ❌ Don't include full narrative/facts/files in index format
---
## Format: Full Results
**When to use:** User requests details, specific items selected from index
**Structure:**
```markdown
## {emoji} {type} #{id}: {title}
**Summary:** {subtitle}
**What happened:**
{narrative}
**Key Facts:**
- {fact1}
- {fact2}
**Files modified:**
- {file1}
- {file2}
**Concepts:** {concepts}
**Date:** {human_readable_date}
**Project:** {project}
```
**Example:**
```markdown
## 🟣 Feature #1234: Implemented JWT authentication
**Summary:** Added token-based auth with refresh tokens
**What happened:**
Implemented a complete JWT authentication system with access and refresh tokens. Access tokens expire after 15 minutes, refresh tokens after 7 days. Added token signing with RS256 algorithm and proper key rotation infrastructure.
**Key Facts:**
- Access tokens use 15-minute expiration
- Refresh tokens stored in httpOnly cookies
- RS256 algorithm with key rotation support
- Token refresh endpoint handles race conditions gracefully
**Files modified:**
- src/auth/jwt.ts (created)
- src/auth/refresh.ts (created)
- src/middleware/auth.ts (modified)
**Concepts:** how-it-works, pattern
**Date:** November 9, 2024 at 2:55 PM
**Project:** api-server
```
**What to include:**
- ✅ Full title with emoji and ID
- ✅ Summary/subtitle
- ✅ Complete narrative
- ✅ All key facts
- ✅ All files (with status: created/modified/deleted)
- ✅ Concepts/tags
- ✅ Precise timestamp
- ✅ Project name
---
## Format: Timeline Results
**When to use:** Temporal investigations, "what was happening" questions
**Structure:**
```markdown
## Timeline: {anchor_description}
### Before ({count} records)
**{time}** - {emoji} {type} #{id}: {title}
**{time}** - {emoji} {type} #{id}: {title}
### ⭐ Anchor Point ({time})
{emoji} **{type} #{id}**: {title}
### After ({count} records)
**{time}** - {emoji} {type} #{id}: {title}
**{time}** - {emoji} {type} #{id}: {title}
```
**Example:**
```markdown
## Timeline: Deployment
### Before (10 records)
**2:30 PM** - 🟣 #1230: Prepared deployment scripts
**2:45 PM** - 🔄 #1232: Updated configuration files
**2:50 PM** - 💬 User asked: "Are we ready to deploy?"
### ⭐ Anchor Point (2:55 PM)
🎯 **Session #545**: Deployed to production
### After (10 records)
**3:00 PM** - 🔴 #1235: Fixed post-deployment routing issue
**3:10 PM** - 🔵 #1236: Discovered caching behavior in production
**3:15 PM** - 🧠 #1237: Decided to add health check endpoint
```
**What to include:**
- ✅ Chronological ordering (oldest to newest)
- ✅ Human-readable times (not epochs)
- ✅ Clear anchor point marker (⭐)
- ✅ Mix of all record types (observations, sessions, prompts)
- ✅ Concise titles (not full narratives)
- ✅ Type emojis for quick scanning
---
## Format: Session Summaries
**When to use:** Recent context, "what did we do" questions
**Structure:**
```markdown
## Recent Work on {project}
### 🎯 Session #{id} - {date}
**Request:** {user_request}
**Completed:**
- {completion1}
- {completion2}
**Key Learning:** {learning}
**Observations:**
- {emoji} **#{obs_id}** {obs_title}
- Files: {file_list}
```
**Example:**
```markdown
## Recent Work on api-server
### 🎯 Session #545 - November 9, 2024
**Request:** Add JWT authentication with refresh tokens
**Completed:**
- Implemented token-based auth with refresh logic
- Added JWT signing and verification
- Created refresh token rotation
**Key Learning:** JWT expiration requires careful handling of refresh race conditions
**Observations:**
- 🟣 **#1234** Implemented JWT authentication
- Files: jwt.ts, refresh.ts, middleware/auth.ts
- 🔴 **#1235** Fixed token expiration edge case
- Files: refresh.ts
```
**What to include:**
- ✅ Session ID and date
- ✅ Original user request
- ✅ What was completed (bulleted list)
- ✅ Key learnings/insights
- ✅ Linked observations with file lists
- ✅ Clear hierarchy (session → observations)
---
## Format: User Prompts
**When to use:** "What did I ask" questions, prompt searches
**Structure:**
```markdown
Found {count} user prompts:
💬 **Prompt #{id}** (Session #{session_id})
> "{preview_text}"
> {date} • {project}
```
**Example:**
```markdown
Found 5 user prompts about "authentication":
💬 **Prompt #1250** (Session #545)
> "How do I implement JWT authentication with refresh tokens? I need to handle token expiration..."
> Nov 9, 2024 • api-server
💬 **Prompt #1251** (Session #546)
> "The auth tokens are expiring too quickly. Can you help debug the refresh flow?"
> Nov 8, 2024 • api-server
```
**What to include:**
- ✅ Prompt ID
- ✅ Session ID (for context linking)
- ✅ Preview text (200 chars for index, full text for full format)
- ✅ Date and project
- ✅ Quote formatting for prompt text
---
## Error Responses
**No results found:**
```markdown
No results found for "{query}". Try:
- Different search terms
- Broader keywords
- Checking spelling
- Using partial paths (for file searches)
```
**Service unavailable:**
```markdown
The search service isn't available. Check if the worker is running:
```bash
pm2 list
```
If the worker is stopped, restart it:
```bash
npm run worker:restart
```
```
**Invalid parameters:**
```markdown
Invalid search parameters:
- {parameter}: {error_message}
See the [API help](help.md) for valid parameter options.
```
---
## Context-Aware Presentation
Tailor formatting to user's question:
**"What bugs did we fix?"**
→ Use index format, emphasize date/type, group by recency
**"How did we implement X?"**
→ Use full format for best match, include complete narrative and files
**"What was happening when..."**
→ Use timeline format, emphasize chronology and causality
**"Catch me up on recent work"**
→ Use session summary format, focus on high-level accomplishments
---
## Token Budget Guidelines
**Minimal presentation (~100-200 tokens):**
- Index format with 3-5 results
- Compact list structure
- Essential metadata only
**Standard presentation (~500-1,000 tokens):**
- Index format with 10-15 results
- Include subtitles and context
- Clear formatting and emojis
**Detailed presentation (~1,500-3,000 tokens):**
- Full format for 2-3 items
- Complete narratives and facts
- Timeline with 20-30 records
**Comprehensive presentation (~5,000+ tokens):**
- Multiple full results
- Deep timelines (40+ records)
- Session summaries with observations
Always start minimal, expand only when needed.
---
## Markdown Best Practices
1. **Use headers (##, ###)** for hierarchy
2. **Bold important elements** (IDs, titles, dates)
3. **Quote user text** (prompts, questions)
4. **Bullet lists** for facts and files
5. **Code blocks** for commands and examples
6. **Emojis** for type indicators
7. **Horizontal rules (---)** for section breaks
8. **Blockquotes (>)** for subtitles and previews
---
## Examples by Use Case
### Use Case 1: Quick Overview
User: "What did we do last session?"
```markdown
## Recent Work
### 🎯 Session #545 - November 9, 2024
Implemented JWT authentication system
**Key accomplishment:** Added token-based auth with refresh tokens
**Key learning:** JWT expiration requires careful handling of refresh race conditions
```
### Use Case 2: Specific Investigation
User: "How did we implement JWT authentication?"
```markdown
## 🟣 Feature #1234: Implemented JWT authentication
**What happened:**
Implemented a complete JWT authentication system with access and refresh tokens. Access tokens expire after 15 minutes, refresh tokens after 7 days. Added token signing with RS256 algorithm.
**Files:**
- src/auth/jwt.ts (created)
- src/auth/refresh.ts (created)
- src/middleware/auth.ts (modified)
**Key insight:** Refresh race conditions require atomic token exchange logic.
```
### Use Case 3: Timeline Investigation
User: "What was happening around the deployment?"
```markdown
## Timeline: Deployment
[... chronological timeline with before/after context ...]
```
Choose presentation style based on user's question and information needs.

View File

@@ -0,0 +1,175 @@
# API Help
Get comprehensive API documentation for all search endpoints.
## When to Use
- User asks: "What search operations are available?"
- User asks: "How do I use the search API?"
- Need reference documentation for endpoints
- Want to see all available parameters
## Command
```bash
curl -s "http://localhost:37777/api/help"
```
## Response Structure
Returns complete API documentation:
```json
{
"version": "5.4.0",
"base_url": "http://localhost:37777/api",
"endpoints": [
{
"path": "/search/observations",
"method": "GET",
"description": "Search observations using full-text search",
"parameters": [
{
"name": "query",
"required": true,
"type": "string",
"description": "Search terms"
},
{
"name": "format",
"required": false,
"type": "string",
"default": "full",
"options": ["index", "full"],
"description": "Response format"
}
],
"example": "curl -s \"http://localhost:37777/api/search/observations?query=authentication&format=index&limit=5\""
}
]
}
```
## How to Present Results
Present as reference documentation:
```markdown
## claude-mem Search API Reference (v5.4.0)
Base URL: `http://localhost:37777/api`
### Search Operations
**1. Search Observations**
- **Endpoint:** `GET /search/observations`
- **Description:** Search observations using full-text search
- **Parameters:**
- `query` (required, string): Search terms
- `format` (optional, string): "index" or "full" (default: "full")
- `limit` (optional, number): Max results (default: 20, max: 100)
- **Example:**
```bash
curl -s "http://localhost:37777/api/search/observations?query=authentication&format=index&limit=5"
```
[... continue for all endpoints ...]
```
## Endpoint Categories
The API help response organizes endpoints by category:
1. **Full-Text Search**
- `/search/observations`
- `/search/sessions`
- `/search/prompts`
2. **Filtered Search**
- `/search/by-type`
- `/search/by-concept`
- `/search/by-file`
3. **Context Retrieval**
- `/context/recent`
- `/timeline/context`
- `/timeline/by-query`
4. **Utilities**
- `/help`
## Common Parameters
Many endpoints share these parameters:
- **format**: "index" (summary) or "full" (complete details)
- **limit**: Number of results to return
- **offset**: Number of results to skip (for pagination)
- **project**: Filter by project name
- **dateRange**: Filter by date range
- `dateRange[start]`: Start date (ISO string or epoch)
- `dateRange[end]`: End date (ISO string or epoch)
## Error Handling
**Worker not running:**
Connection refused error. Response: "The search API isn't available. Check if worker is running: `pm2 list`"
**Invalid endpoint:**
```json
{"error": "Not found"}
```
Response: "Invalid API endpoint. Use /api/help to see available endpoints."
## Tips
1. Save help response for reference during investigation
2. Use examples as starting point for your queries
3. Check required parameters before making requests
4. Refer to format options for each endpoint
5. All endpoints use GET method with query parameters
**Token Efficiency:**
- Help response: ~2,000-3,000 tokens (complete API reference)
- Use sparingly - refer to operation-specific docs instead
- Keep help response cached for repeated reference
## When to Use Help
**Use help when:**
- Starting to use the search API
- Need complete parameter reference
- Forgot which endpoints are available
- Want to see all options at once
**Don't use help when:**
- You know which operation you need (use operation-specific docs)
- Just need examples (use common-workflows.md)
- Token budget is limited (help is comprehensive)
## Alternative to Help Endpoint
Instead of calling `/api/help`, you can:
1. **Use SKILL.md** - Quick decision guide with operation links
2. **Use operation docs** - Detailed guides for specific endpoints
3. **Use common-workflows.md** - Step-by-step examples
4. **Use formatting.md** - Response presentation templates
The help endpoint is most useful when you need complete API reference in one response.
## API Versioning
The help response includes version information:
```json
{
"version": "5.4.0",
"skill_migration": true,
"deprecated": {
"mcp_tools": "Replaced by HTTP API in v5.4.0"
}
}
```
Check version to ensure compatibility with documentation.

View File

@@ -0,0 +1,124 @@
# Search Observations (Semantic + Full-Text Hybrid)
Search all observations using natural language queries.
## When to Use
- User asks: "How did we implement authentication?"
- User asks: "What bugs did we fix?"
- User asks: "What features did we add?"
- Looking for past work by keyword or topic
## Command
```bash
curl -s "http://localhost:37777/api/search/observations?query=authentication&format=index&limit=5"
```
## Parameters
- **query** (optional): Natural language search query - uses semantic search (ChromaDB) for ranking with SQLite FTS5 fallback (e.g., "authentication", "bug fix", "database migration"). Can be omitted for filter-only searches.
- **format**: "index" (summary) or "full" (complete details). Default: "full"
- **limit**: Number of results (default: 20, max: 100)
- **project**: Filter by project name (optional)
- **dateRange**: Filter by date range (optional) - `dateRange[start]` and/or `dateRange[end]`
- **obs_type**: Filter by observation type: bugfix, feature, refactor, decision, discovery, change (optional)
- **concepts**: Filter by concept tags (optional)
- **files**: Filter by file paths (optional)
**Important**: When omitting `query`, you MUST provide at least one filter (project, dateRange, obs_type, concepts, or files)
## When to Use Each Format
**Use format=index for:**
- Quick overviews
- Finding IDs for deeper investigation
- Listing multiple results
- **Token cost: ~50-100 per result**
**Use format=full for:**
- Complete details including narrative, facts, files, concepts
- Understanding the full context of specific observations
- **Token cost: ~500-1000 per result**
## Example Response (format=index)
```json
{
"query": "authentication",
"count": 5,
"format": "index",
"results": [
{
"id": 1234,
"type": "feature",
"title": "Implemented JWT authentication",
"subtitle": "Added token-based auth with refresh tokens",
"created_at_epoch": 1699564800000,
"project": "api-server",
"score": 0.95
}
]
}
```
## How to Present Results
For format=index, present as a compact list:
```markdown
Found 5 results for "authentication":
1. **#1234** [feature] Implemented JWT authentication
> Added token-based auth with refresh tokens
> Nov 9, 2024 • api-server
2. **#1235** [bugfix] Fixed token expiration edge case
> Handled race condition in refresh flow
> Nov 9, 2024 • api-server
```
**Include:** ID (for follow-up), type emoji (🔴 bugfix, 🟣 feature, 🔄 refactor, 🔵 discovery, 🧠 decision, ✅ change), title, subtitle, date, project.
For complete formatting guidelines, see formatting.md (documentation coming soon).
## Filter-Only Examples
Search without query text (direct SQLite filtering):
```bash
# Get all observations from November 2025
curl -s "http://localhost:37777/api/search?type=observations&dateRange[start]=2025-11-01&format=index"
# Get all bug fixes from a specific project
curl -s "http://localhost:37777/api/search?type=observations&obs_type=bugfix&project=api-server&format=index"
# Get all observations from last 7 days
curl -s "http://localhost:37777/api/search?type=observations&dateRange[start]=2025-11-11&format=index"
```
## Error Handling
**Missing query and filters:**
```json
{"error": "Either query or filters required for search"}
```
Fix: Provide either a query parameter OR at least one filter (project, dateRange, obs_type, concepts, files)
**No results found:**
```json
{"query": "foobar", "count": 0, "results": []}
```
Response: "No results found for 'foobar'. Try different search terms."
## Tips
1. Be specific: "authentication JWT" > "auth"
2. Start with format=index and limit=5-10
3. Use project filtering when working on one codebase
4. If no results, try broader terms or check spelling
**Token Efficiency:**
- Start with format=index (~50-100 tokens per result)
- Use format=full only for relevant items (~500-1000 tokens per result)
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)

View File

@@ -0,0 +1,125 @@
# Search User Prompts (Full-Text)
Search raw user prompts to find what was actually asked across all sessions.
## When to Use
- User asks: "What did I ask about authentication?"
- User asks: "Find my question about database migrations"
- User asks: "When did I ask about testing?"
- Looking for specific user questions or requests
## Command
```bash
curl -s "http://localhost:37777/api/search/prompts?query=authentication&format=index&limit=5"
```
## Parameters
- **query** (required): Search terms (e.g., "authentication", "how do I", "bug fix")
- **format**: "index" (truncated prompts) or "full" (complete prompt text). Default: "full"
- **limit**: Number of results (default: 20, max: 100)
- **project**: Filter by project name (optional)
- **dateRange**: Filter by date range (optional)
## When to Use Each Format
**Use format=index for:**
- Quick overviews of what was asked
- Finding prompt IDs for full text
- Listing multiple prompts
- **Token cost: ~50-100 per result (truncated to 200 chars)**
**Use format=full for:**
- Complete prompt text
- Understanding the full user request
- **Token cost: Variable (depends on prompt length, typically 100-300 tokens)**
## Example Response (format=index)
```json
{
"query": "authentication",
"count": 5,
"format": "index",
"results": [
{
"id": 1250,
"session_id": "S545",
"prompt_preview": "How do I implement JWT authentication with refresh tokens? I need to handle token expiration...",
"created_at_epoch": 1699564800000,
"project": "api-server"
}
]
}
```
## How to Present Results
For format=index, present as a compact list:
```markdown
Found 5 user prompts about "authentication":
💬 **Prompt #1250** (Session #545)
> "How do I implement JWT authentication with refresh tokens? I need to handle token expiration..."
> Nov 9, 2024 • api-server
💬 **Prompt #1251** (Session #546)
> "The auth tokens are expiring too quickly. Can you help debug the refresh flow?"
> Nov 8, 2024 • api-server
```
For complete formatting guidelines, see [formatting.md](formatting.md).
## What Gets Searched
User prompts search covers:
- All user messages sent to Claude Code
- Raw text as typed by the user
- Multi-turn conversations (each message is a separate prompt)
- Questions, requests, commands, and clarifications
## Error Handling
**Missing query parameter:**
```json
{"error": "Missing required parameter: query"}
```
Fix: Add the query parameter
**No results found:**
```json
{"query": "foobar", "count": 0, "results": []}
```
Response: "No user prompts found for 'foobar'. Try different search terms."
## Tips
1. Use exact phrases in quotes: `?query="how do I"` for precise matches
2. Start with format=index to see preview, then get full text if needed
3. Use dateRange to find recent questions: `?query=bug&dateRange[start]=2024-11-01`
4. Prompts show what was asked, sessions/observations show what was done
5. Combine with session search to see both question and answer
**Token Efficiency:**
- Start with format=index (~50-100 tokens per result, prompt truncated to 200 chars)
- Use format=full only for relevant items (100-300 tokens per result)
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
## When to Use Prompts vs Sessions
**Use prompts search when:**
- Looking for specific user questions
- Trying to remember what was asked
- Finding original request wording
**Use sessions search when:**
- Looking for what was accomplished
- Understanding work summaries
- Getting high-level context
**Combine both when:**
- Understanding the full conversation (what was asked + what was done)
- Investigating how a request was interpreted

View File

@@ -0,0 +1,134 @@
# Get Recent Context
Get recent session summaries and observations for a project.
## When to Use
- User asks: "What did we do last session?"
- User asks: "What have we been working on recently?"
- User asks: "Catch me up on recent work"
- Starting a new session and need context
## Command
```bash
curl -s "http://localhost:37777/api/context/recent?project=api-server&limit=3"
```
## Parameters
- **project**: Project name (defaults to current working directory basename)
- **limit**: Number of recent sessions to retrieve (default: 3, max: 10)
## Response Structure
Returns combined context from recent sessions:
```json
{
"project": "api-server",
"limit": 3,
"sessions": [
{
"id": 545,
"session_id": "S545",
"title": "Implemented JWT authentication system",
"request": "Add JWT authentication with refresh tokens",
"completion": "Implemented token-based auth with refresh logic",
"learnings": "JWT expiration requires careful handling of refresh race conditions",
"created_at_epoch": 1699564800000,
"observations": [
{
"id": 1234,
"type": "feature",
"title": "Implemented JWT authentication",
"subtitle": "Added token-based auth with refresh tokens",
"files": ["src/auth/jwt.ts", "src/auth/refresh.ts"]
}
]
}
]
}
```
## How to Present Results
Present as a chronological narrative:
```markdown
## Recent Work on api-server
### Session #545 - Nov 9, 2024
**Request:** Add JWT authentication with refresh tokens
**Completed:**
- Implemented token-based auth with refresh logic
- Added JWT signing and verification
- Created refresh token rotation
**Key Learning:** JWT expiration requires careful handling of refresh race conditions
**Observations:**
- 🟣 **#1234** Implemented JWT authentication
- Files: jwt.ts, refresh.ts
```
For complete formatting guidelines, see [formatting.md](formatting.md).
## Default Project Detection
If no project parameter is provided, uses current working directory:
```bash
# Auto-detects project from current directory
curl -s "http://localhost:37777/api/context/recent?limit=3"
```
## Error Handling
**No sessions found:**
```json
{"project": "new-project", "sessions": []}
```
Response: "No recent sessions found for 'new-project'. This might be a new project."
**Worker not running:**
Connection refused error. Inform user to check if worker is running: `pm2 list`
## Tips
1. Start with limit=3 for quick overview (default)
2. Increase to limit=5-10 for deeper context
3. Recent context is perfect for session start
4. Combines both sessions and observations in one request
5. Use this when user asks "what did we do last time?"
**Token Efficiency:**
- limit=3 sessions: ~1,500-2,500 tokens (includes observations)
- limit=5 sessions: ~2,500-4,000 tokens
- limit=10 sessions: ~5,000-8,000 tokens
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
## When to Use Recent Context
**Use recent-context when:**
- Starting a new session
- User asks about recent work
- Need quick catch-up on project activity
- Want both sessions and observations together
**Don't use recent-context when:**
- Looking for specific topics (use search instead)
- Need timeline around specific event (use timeline instead)
- Want only observations or only sessions (use search operations)
## Comparison with Other Operations
| Operation | Use Case | Token Cost |
|-----------|----------|------------|
| recent-context | Quick catch-up on recent work | 1,500-4,000 |
| sessions search | Find sessions by topic | 50-100 per result (index) |
| observations search | Find specific implementations | 50-100 per result (index) |
| timeline | Context around specific point | 3,000-6,000 |
Recent context is optimized for "what happened recently?" questions with minimal token usage.

View File

@@ -0,0 +1,124 @@
# Search Sessions (Full-Text)
Search session summaries using natural language queries.
## When to Use
- User asks: "What did we work on last week?"
- User asks: "What sessions involved database work?"
- User asks: "Show me sessions where we fixed bugs"
- Looking for past sessions by topic or theme
## Command
```bash
curl -s "http://localhost:37777/api/search/sessions?query=authentication&format=index&limit=5"
```
## Parameters
- **query** (required): Search terms (e.g., "authentication", "database migration", "bug fixes")
- **format**: "index" (summary) or "full" (complete details). Default: "full"
- **limit**: Number of results (default: 20, max: 100)
- **project**: Filter by project name (optional)
- **dateRange**: Filter by date range (optional)
## When to Use Each Format
**Use format=index for:**
- Quick overviews of past sessions
- Finding session IDs for deeper investigation
- Listing multiple sessions
- **Token cost: ~50-100 per result**
**Use format=full for:**
- Complete session summaries with requests, completions, learnings
- Understanding the full context of a session
- **Token cost: ~500-1000 per result**
## Example Response (format=index)
```json
{
"query": "authentication",
"count": 3,
"format": "index",
"results": [
{
"id": 545,
"session_id": "S545",
"title": "Implemented JWT authentication system",
"subtitle": "Added token-based auth with refresh tokens",
"created_at_epoch": 1699564800000,
"project": "api-server"
}
]
}
```
## How to Present Results
For format=index, present as a compact list:
```markdown
Found 3 sessions about "authentication":
🎯 **Session #545** Implemented JWT authentication system
> Added token-based auth with refresh tokens
> Nov 9, 2024 • api-server
🎯 **Session #546** Fixed authentication token expiration
> Resolved race condition in token refresh flow
> Nov 8, 2024 • api-server
```
For complete formatting guidelines, see [formatting.md](formatting.md).
## Session Summary Structure
Full session summaries include:
- **Session request**: What the user asked for
- **What was completed**: Summary of work done
- **Key learnings**: Important insights and discoveries
- **Files modified**: List of changed files
- **Observations**: Links to detailed observations
## Error Handling
**Missing query parameter:**
```json
{"error": "Missing required parameter: query"}
```
Fix: Add the query parameter
**No results found:**
```json
{"query": "foobar", "count": 0, "results": []}
```
Response: "No sessions found for 'foobar'. Try different search terms."
## Tips
1. Be specific: "JWT authentication implementation" > "auth"
2. Start with format=index and limit=5-10
3. Use dateRange for recent sessions: `?query=auth&dateRange[start]=2024-11-01`
4. Sessions provide high-level overview, observations provide details
5. Use project filtering when working on one codebase
**Token Efficiency:**
- Start with format=index (~50-100 tokens per result)
- Use format=full only for relevant items (~500-1000 tokens per result)
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
## When to Use Sessions vs Observations
**Use sessions search when:**
- Looking for high-level work summaries
- Understanding what was done in past sessions
- Getting overview of recent activity
**Use observations search when:**
- Looking for specific implementation details
- Finding bugs, features, or decisions
- Need fine-grained context about code changes

View File

@@ -0,0 +1,192 @@
# Timeline by Query
Search for observations and get timeline context in a single request. Combines search + timeline into one operation.
## When to Use
- User asks: "What was happening when we worked on authentication?"
- User asks: "Show me context around bug fixes"
- User asks: "Timeline of database work"
- Need to find something then see temporal context
## Command
```bash
# Auto mode: Uses top search result as timeline anchor
curl -s "http://localhost:37777/api/timeline/by-query?query=authentication&mode=auto&depth_before=10&depth_after=10"
# Interactive mode: Shows top N search results for manual selection
curl -s "http://localhost:37777/api/timeline/by-query?query=authentication&mode=interactive&limit=5"
```
## Parameters
- **query** (required): Search terms (e.g., "authentication", "bug fix", "database")
- **mode**: Search mode
- `auto` (default): Automatically uses top search result as timeline anchor
- `interactive`: Returns top N search results for manual anchor selection
- **depth_before**: Records before anchor (default: 10, max: 50) - for auto mode
- **depth_after**: Records after anchor (default: 10, max: 50) - for auto mode
- **limit**: Number of search results (default: 5, max: 20) - for interactive mode
- **project**: Filter by project name (optional)
## Auto Mode (Recommended)
Automatically gets timeline around best match:
```bash
curl -s "http://localhost:37777/api/timeline/by-query?query=JWT+authentication&mode=auto&depth_before=10&depth_after=10"
```
**Response:**
```json
{
"query": "JWT authentication",
"mode": "auto",
"best_match": {
"id": 1234,
"type": "feature",
"title": "Implemented JWT authentication",
"score": 0.95
},
"timeline": [
// ... timeline records around observation #1234
]
}
```
**When to use auto mode:**
- You're confident the top result is what you want
- Want fastest path to timeline context
- Query is specific enough for accurate top result
## Interactive Mode
Shows top search results for manual review:
```bash
curl -s "http://localhost:37777/api/timeline/by-query?query=authentication&mode=interactive&limit=5"
```
**Response:**
```json
{
"query": "authentication",
"mode": "interactive",
"top_matches": [
{
"id": 1234,
"type": "feature",
"title": "Implemented JWT authentication",
"subtitle": "Added token-based auth with refresh tokens",
"score": 0.95
},
{
"id": 1240,
"type": "bugfix",
"title": "Fixed authentication token expiration",
"subtitle": "Resolved race condition in refresh flow",
"score": 0.87
}
],
"next_step": "Use /api/timeline/context?anchor=<id>&depth_before=10&depth_after=10"
}
```
**When to use interactive mode:**
- Query is broad and may have multiple relevant results
- Want to review options before getting timeline
- Not sure which result is most relevant
## How to Present Results
**For auto mode:**
```markdown
## Timeline: JWT authentication
**Best Match:** 🟣 Observation #1234 - Implemented JWT authentication (score: 0.95)
### Before (10 records)
**2:45 PM** - 🟣 Added authentication middleware
### ⭐ Anchor Point (2:55 PM)
🟣 **Observation #1234**: Implemented JWT authentication
### After (10 records)
**3:00 PM** - 🎯 Session completed: JWT authentication system
```
**For interactive mode:**
```markdown
Found 5 matches for "authentication":
1. 🟣 **#1234** Implemented JWT authentication (score: 0.95)
> Added token-based auth with refresh tokens
2. 🔴 **#1240** Fixed authentication token expiration (score: 0.87)
> Resolved race condition in refresh flow
To see timeline context, use observation ID with timeline operation.
```
For complete formatting guidelines, see [formatting.md](formatting.md).
## Error Handling
**Missing query parameter:**
```json
{"error": "Missing required parameter: query"}
```
Fix: Add the query parameter
**No results found:**
```json
{"query": "foobar", "top_matches": []}
```
Response: "No results found for 'foobar'. Try different search terms."
## Tips
1. **Use auto mode** for specific queries: "JWT authentication implementation"
2. **Use interactive mode** for broad queries: "authentication"
3. Start with depth 10/10 for balanced context
4. Be specific in queries for better auto mode accuracy
5. This is fastest way to find + explore context in one request
**Token Efficiency:**
- Auto mode: ~3,000-4,000 tokens (search + timeline)
- Interactive mode: ~500-1,000 tokens (search results only)
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
## Workflow Comparison
**timeline-by-query (auto):**
1. One request → get timeline around best match
2. ~3,000 tokens
**timeline-by-query (interactive) → timeline:**
1. First request → see top matches (~500 tokens)
2. Second request → get timeline for chosen match (~3,000 tokens)
3. Total: ~3,500 tokens
**observations search → timeline:**
1. Search observations (~500 tokens)
2. Get timeline for chosen result (~3,000 tokens)
3. Total: ~3,500 tokens
Use auto mode when you're confident about the query. Use interactive mode or separate search when you want more control.
## When to Use Timeline-by-Query
**Use timeline-by-query when:**
- Need to find something AND see temporal context
- Want one-request convenience (auto mode)
- Investigating "what was happening when we worked on X?"
- Don't have observation ID already
**Don't use timeline-by-query when:**
- Already have observation ID (use timeline instead)
- Just need search results (use observations search)
- Need recent work overview (use recent-context)

View File

@@ -0,0 +1,174 @@
# Get Context Timeline
Get a chronological timeline of observations, sessions, and prompts around a specific point in time.
## When to Use
- User asks: "What was happening when we deployed?"
- User asks: "Show me context around that bug fix"
- User asks: "What happened before and after that change?"
- Need temporal context around an event
## Command
```bash
# Using observation ID as anchor
curl -s "http://localhost:37777/api/timeline/context?anchor=1234&depth_before=10&depth_after=10"
# Using session ID as anchor
curl -s "http://localhost:37777/api/timeline/context?anchor=S545&depth_before=10&depth_after=10"
# Using ISO timestamp as anchor
curl -s "http://localhost:37777/api/timeline/context?anchor=2024-11-09T12:00:00Z&depth_before=10&depth_after=10"
```
## Parameters
- **anchor** (required): Point in time to center timeline
- Observation ID: `1234`
- Session ID: `S545`
- ISO timestamp: `2024-11-09T12:00:00Z`
- **depth_before**: Number of records before anchor (default: 10, max: 50)
- **depth_after**: Number of records after anchor (default: 10, max: 50)
- **project**: Filter by project name (optional)
## Response Structure
Returns unified chronological timeline:
```json
{
"anchor": 1234,
"depth_before": 10,
"depth_after": 10,
"total_records": 21,
"timeline": [
{
"record_type": "observation",
"id": 1230,
"type": "feature",
"title": "Added authentication middleware",
"created_at_epoch": 1699564700000
},
{
"record_type": "prompt",
"id": 1250,
"session_id": "S545",
"prompt_preview": "How do I add JWT authentication?",
"created_at_epoch": 1699564750000
},
{
"record_type": "observation",
"id": 1234,
"type": "feature",
"title": "Implemented JWT authentication",
"created_at_epoch": 1699564800000,
"is_anchor": true
},
{
"record_type": "session",
"id": 545,
"session_id": "S545",
"title": "Implemented JWT authentication system",
"created_at_epoch": 1699564900000
}
]
}
```
## How to Present Results
Present as chronological narrative with anchor highlighted:
```markdown
## Timeline around Observation #1234
### Before (10 records)
**2:45 PM** - 🟣 Observation #1230: Added authentication middleware
**2:50 PM** - 💬 User asked: "How do I add JWT authentication?"
### ⭐ Anchor Point (2:55 PM)
🟣 **Observation #1234**: Implemented JWT authentication
### After (10 records)
**3:00 PM** - 🎯 Session #545 completed: Implemented JWT authentication system
**3:05 PM** - 🔴 Observation #1235: Fixed token expiration edge case
```
For complete formatting guidelines, see [formatting.md](formatting.md).
## Anchor Types
**Observation ID:**
- Use when you know the specific observation ID
- Example: `anchor=1234`
**Session ID:**
- Use when you want context around a session
- Example: `anchor=S545`
**ISO Timestamp:**
- Use when you know approximate time
- Example: `anchor=2024-11-09T14:30:00Z`
## Error Handling
**Missing anchor parameter:**
```json
{"error": "Missing required parameter: anchor"}
```
Fix: Add the anchor parameter
**Anchor not found:**
```json
{"error": "Anchor not found: 9999"}
```
Response: "Observation #9999 not found. Check the ID or try a different anchor."
**Invalid timestamp:**
```json
{"error": "Invalid timestamp format"}
```
Fix: Use ISO 8601 format: `2024-11-09T14:30:00Z`
## Tips
1. Start with depth_before=10, depth_after=10 for balanced context
2. Increase depth for broader investigation (max: 50 each)
3. Use observation IDs from search results as anchors
4. Timelines show all record types interleaved chronologically
5. Perfect for understanding "what was happening when X occurred"
**Token Efficiency:**
- depth 10/10: ~3,000-4,000 tokens (21 records)
- depth 20/20: ~6,000-8,000 tokens (41 records)
- depth 50/50: ~15,000-20,000 tokens (101 records)
- See [../principles/progressive-disclosure.md](../principles/progressive-disclosure.md)
## When to Use Timeline
**Use timeline when:**
- Need context around specific event
- Understanding sequence of events
- Investigating "what was happening then?"
- Want all record types (observations, sessions, prompts) together
**Don't use timeline when:**
- Just need recent work (use recent-context)
- Looking for specific topics (use search)
- Don't have an anchor point (use timeline-by-query)
## Comparison with Timeline-by-Query
| Feature | timeline | timeline-by-query |
|---------|----------|-------------------|
| Requires anchor | Yes (ID or timestamp) | No (uses search query) |
| Best for | Known event investigation | Finding then exploring context |
| Steps | 1 (direct timeline) | 2 (search + timeline) |
| Use when | You have observation ID | You have search term |
Timeline is faster when you already know the anchor point.

View File

View File

@@ -0,0 +1,176 @@
# Anti-Pattern Catalogue
Common mistakes to avoid when using the HTTP search API. These anti-patterns address LLM training biases and prevent token-wasting behaviors.
## Anti-Pattern 1: Skipping Index Format
**The Mistake:**
```bash
# ❌ Bad: Jump straight to full format
curl -s "http://localhost:37777/api/search/observations?query=authentication&format=full&limit=20"
```
**Why It's Wrong:**
- 20 × 750 tokens = 15,000 tokens
- May hit MCP token limits
- 99% wasted on irrelevant results
**The Correction:**
```bash
# ✅ Good: Start with index, review, then request full selectively
curl -s "http://localhost:37777/api/search/observations?query=authentication&format=index&limit=5"
# Review results, identify relevant items
curl -s "http://localhost:37777/api/search/observations?query=authentication&format=full&limit=1&offset=2"
```
**What It Teaches:**
Progressive disclosure isn't optional - it's essential for scale.
**LLM Behavior Insight:**
LLMs trained on code examples may have seen `format=full` as "more complete" and default to it.
---
## Anti-Pattern 2: Over-Requesting Results
**The Mistake:**
```bash
# ❌ Bad: Request limit=20 without reviewing index first
curl -s "http://localhost:37777/api/search/observations?query=auth&format=index&limit=20"
```
**Why It's Wrong:**
- Most of 20 results will be irrelevant
- Wastes tokens and time
- Overwhelms review process
**The Correction:**
```bash
# ✅ Good: Start small, paginate if needed
curl -s "http://localhost:37777/api/search/observations?query=auth&format=index&limit=5"
# If needed, paginate:
curl -s "http://localhost:37777/api/search/observations?query=auth&format=index&limit=5&offset=5"
```
**What It Teaches:**
Start small (limit=3-5), review, paginate if needed.
**LLM Behavior Insight:**
LLMs may think "more results = more thorough" without considering relevance.
---
## Anti-Pattern 3: Ignoring Tool Specialization
**The Mistake:**
```bash
# ❌ Bad: Use generic search for everything
curl -s "http://localhost:37777/api/search/observations?query=bugfix&format=index&limit=10"
```
**Why It's Wrong:**
- Specialized tools (by-type, by-concept, by-file) are more efficient
- Generic search mixes all result types
- Misses filtering optimization
**The Correction:**
```bash
# ✅ Good: Use specialized endpoint when applicable
curl -s "http://localhost:37777/api/search/by-type?type=bugfix&format=index&limit=10"
```
**What It Teaches:**
The decision tree exists for a reason - follow it.
**LLM Behavior Insight:**
LLMs may gravitate toward "general purpose" tools to avoid decision-making.
---
## Anti-Pattern 4: Loading Full Context Prematurely
**The Mistake:**
```bash
# ❌ Bad: Request full format before understanding what's relevant
curl -s "http://localhost:37777/api/search/observations?query=database&format=full&limit=10"
```
**Why It's Wrong:**
- Can't filter relevance without seeing index first
- Wastes tokens on irrelevant full details
- 10 × 750 = 7,500 tokens for potentially zero useful results
**The Correction:**
```bash
# ✅ Good: Index first to identify relevance
curl -s "http://localhost:37777/api/search/observations?query=database&format=index&limit=10"
# Identify relevant: #1234 and #1250
curl -s "http://localhost:37777/api/search/observations?query=database+1234&format=full&limit=1"
curl -s "http://localhost:37777/api/search/observations?query=database+1250&format=full&limit=1"
```
**What It Teaches:**
Filtering is a prerequisite for expansion.
**LLM Behavior Insight:**
LLMs may try to "get everything at once" to avoid multiple tool calls.
---
## Anti-Pattern 5: Not Using Timeline Tools
**The Mistake:**
```bash
# ❌ Bad: Search for individual observations separately
curl -s "http://localhost:37777/api/search/observations?query=before+deployment"
curl -s "http://localhost:37777/api/search/observations?query=during+deployment"
curl -s "http://localhost:37777/api/search/observations?query=after+deployment"
```
**Why It's Wrong:**
- Misses context around events
- Inefficient (N searches vs 1 timeline)
- Temporal relationships lost
**The Correction:**
```bash
# ✅ Good: Use timeline tool for contextual investigation
curl -s "http://localhost:37777/api/timeline/by-query?query=deployment&depth_before=10&depth_after=10"
```
**What It Teaches:**
Tool composition - some tools are designed to work together.
**LLM Behavior Insight:**
LLMs may not naturally discover tool composition patterns.
---
## Why These Anti-Patterns Matter
**Addresses LLM Training Bias:**
LLMs default to "load everything" behavior from web scraping training data where thoroughness was rewarded.
**Teaches Protocol Awareness:**
HTTP APIs and MCP have real token limits that can break the system.
**Prevents User Frustration:**
Token limit errors confuse users and break workflows.
**Builds Good Habits:**
Anti-patterns teach the "why" behind best practices.
**Makes Implicit Explicit:**
Surfaces mental models that experienced users internalize but novices miss.
---
## What Happens If These Are Ignored
- **No progressive disclosure**: Every search loads limit=20 in full format → token exhaustion
- **Over-requesting**: 15,000 token searches for 2 relevant results
- **Wrong tool**: Generic search when specialized filters would be 10x faster
- **Premature expansion**: Load full details before knowing relevance
- **Missing composition**: Single-tool thinking, missing powerful multi-step workflows
**Bottom Line:** These anti-patterns waste 5-10x more tokens than necessary and frequently cause system failures.

View File

@@ -0,0 +1,120 @@
# Progressive Disclosure Pattern (MANDATORY)
**Core Principle**: Find the smallest set of high-signal tokens first (index format), then drill down to full details only for relevant items.
## The 4-Step Workflow
### Step 1: Start with Index Format
**Action:**
- Use `format=index` (default in most operations)
- Set `limit=3-5` (not 20)
- Review titles and dates ONLY
**Token Cost:** ~50-100 tokens per result
**Why:** Minimal token investment for maximum signal. Get overview before committing to full details.
**Example:**
```bash
curl -s "http://localhost:37777/api/search/observations?query=authentication&format=index&limit=5"
```
**Response:**
```json
{
"query": "authentication",
"count": 5,
"format": "index",
"results": [
{
"id": 1234,
"type": "feature",
"title": "Implemented JWT authentication",
"subtitle": "Added token-based auth with refresh tokens",
"created_at_epoch": 1699564800000,
"project": "api-server"
}
]
}
```
### Step 2: Identify Relevant Items
**Cognitive Task:**
- Scan index results for relevance
- Note which items need full details
- Discard irrelevant items
**Why:** Human-in-the-loop filtering before expensive operations. Don't load full details for items you'll ignore.
### Step 3: Request Full Details (Selectively)
**Action:**
- Use `format=full` ONLY for specific items of interest
- Target by ID or use refined search query
**Token Cost:** ~500-1000 tokens per result
**Principle:** Load only what you need
**Example:**
```bash
# After reviewing index, get full details for observation #1234
curl -s "http://localhost:37777/api/search/observations?query=authentication&format=full&limit=1&offset=2"
```
**Why:** Targeted token expenditure with high ROI. 10x cost difference means selectivity matters.
### Step 4: Refine with Filters (If Needed)
**Techniques:**
- Use `type`, `dateRange`, `concepts`, `files` filters
- Narrow scope BEFORE requesting more results
- Use `offset` for pagination instead of large limits
**Why:** Reduce result set first, then expand selectively. Don't load 20 results when filters could narrow to 3.
## Token Budget Awareness
**Costs:**
- Index result: ~50-100 tokens
- Full result: ~500-1000 tokens
- 10x cost difference
**Starting Points:**
- Start with `limit=3-5` (not 20)
- Reduce limit if hitting token errors
**Savings Example:**
- Naive: 10 items × 750 tokens (avg full) = 7,500 tokens
- Progressive: (5 items × 75 tokens index) + (2 items × 750 tokens full) = 1,875 tokens
- **Savings: 5,625 tokens (75% reduction)**
## What Problems This Solves
1. **Token exhaustion**: Without this, LLMs load everything in full format (9,000+ tokens for 10 items)
2. **Poor signal-to-noise**: Loading full details for irrelevant items wastes tokens
3. **MCP limits**: Large payloads hit protocol limits (system failures)
4. **Inefficiency**: Loading 20 full results when only 2 are relevant
## How It Scales
**With 10 records:**
- Index (500 tokens) → Full (2,000 tokens for 2 relevant) = 2,500 tokens
- Without pattern: Full (10,000 tokens for all 10) = 4x more expensive
**With 1,000 records:**
- Index (500 tokens for top 5) → Full (1,000 tokens for 1 relevant) = 1,500 tokens
- Without pattern: Would hit MCP limits before seeing relevant data
## Context Engineering Alignment
This pattern implements core context engineering principles:
- **Just-in-time context**: Load data dynamically at runtime
- **Progressive disclosure**: Lightweight identifiers (index) → full details as needed
- **Token efficiency**: Minimal high-signal tokens first, expand selectively
- **Attention budget**: Treat context as finite resource with diminishing returns
Always start with the smallest set of high-signal tokens that maximize likelihood of desired outcome.