Initial commit
This commit is contained in:
Zhongwei Li
114
commands/mood-direct.md
Normal file
114
commands/mood-direct.md
Normal file
@@ -0,0 +1,114 @@
|
||||
---
|
||||
description: Direct mood-based movements via daemon API (no conversation app required)
|
||||
---
|
||||
|
||||
# Reachy Mini Mood Plugin (Direct Mode)
|
||||
|
||||
**Lightweight mood system that calls daemon API directly without requiring the conversation app.**
|
||||
|
||||
## Difference from `/reachy-mini:mood`
|
||||
|
||||
**`/reachy-mini:mood`** - Full integration with conversation app (requires run_reachy_pi.py)
|
||||
|
||||
**`/reachy-mini:mood-direct`** - Direct daemon API calls (only requires daemon running)
|
||||
|
||||
## When to Use This
|
||||
|
||||
Use **mood-direct** when:
|
||||
- Testing mood behaviors without full conversation app
|
||||
- Daemon is running but conversation app is not
|
||||
- You want simpler, more predictable mood triggering
|
||||
- Debugging mood system behavior
|
||||
|
||||
Use **mood** (regular) when:
|
||||
- Full conversation app is running
|
||||
- You want synchronized breathing/face tracking integration
|
||||
- Running the complete Reachy Mini conversation system
|
||||
|
||||
## How It Works
|
||||
|
||||
This command uses the plugin's `mood_extractor.py` hook which:
|
||||
|
||||
1. Extracts `<!-- MOOD: mood_name -->` marker from your response
|
||||
2. Polls TTS server status endpoint at `http://localhost:5001/status`
|
||||
3. Continuously plays random emotions from that mood category
|
||||
4. Stops when TTS finishes (`is_playing: false`)
|
||||
5. Falls back to "thoughtful" mood if invalid mood specified
|
||||
|
||||
**Marker Format:**
|
||||
```html
|
||||
<!-- MOOD: mood_name -->
|
||||
```
|
||||
|
||||
## Available Moods (9 Categories)
|
||||
|
||||
### celebratory
|
||||
**Emotions:** success1, success2, proud1-3, cheerful1, electric1, enthusiastic1-2, grateful1, yes1, laughing1-2
|
||||
|
||||
### thoughtful
|
||||
**Emotions:** thoughtful1-2, curious1, attentive1-2, inquiring1-3, understanding1-2
|
||||
|
||||
### welcoming
|
||||
**Emotions:** welcoming1-2, helpful1-2, loving1, come1, grateful1, cheerful1, calming1
|
||||
|
||||
### confused
|
||||
**Emotions:** confused1, uncertain1, lost1, inquiring1-2, incomprehensible2, uncomfortable1, oops1-2
|
||||
|
||||
### frustrated
|
||||
**Emotions:** frustrated1, irritated1-2, impatient1-2, exhausted1, tired1, displeased1-2
|
||||
|
||||
### surprised
|
||||
**Emotions:** surprised1-2, amazed1, oops1-2, incomprehensible2, electric1
|
||||
|
||||
### calm
|
||||
**Emotions:** calming1, serenity1, relief1-2, shy1, understanding1-2, sleep1
|
||||
|
||||
### energetic
|
||||
**Emotions:** electric1, enthusiastic1-2, dance1-3, laughing1-2, yes1, come1
|
||||
|
||||
### playful
|
||||
**Emotions:** laughing1-2, dance1-3, cheerful1, enthusiastic1, oops1-2
|
||||
|
||||
## Usage Example
|
||||
|
||||
```markdown
|
||||
<!-- TTS: "Fixed the bug! The validation middleware now handles edge cases properly." -->
|
||||
<!-- MOOD: celebratory -->
|
||||
|
||||
Fixed! The validation middleware now checks all edge cases. Tests passing, ready for review.
|
||||
```
|
||||
|
||||
## Technical Details
|
||||
|
||||
**Requirements:**
|
||||
- Reachy Mini daemon running (`http://localhost:8100`)
|
||||
- TTS server running (`http://localhost:5001`)
|
||||
|
||||
**Behavior:**
|
||||
- Move timing: 1-2 second intervals between moves
|
||||
- Safety timeout: 60 seconds maximum
|
||||
- Fallback mood: "thoughtful" (if invalid mood specified)
|
||||
- Dataset: `pollen-robotics/reachy-mini-emotions-library`
|
||||
|
||||
**No Integration With:**
|
||||
- Conversation app breathing system
|
||||
- Face tracking synchronization
|
||||
- External control state management
|
||||
|
||||
## Quick Decision Guide
|
||||
|
||||
| **Situation** | **Mood** |
|
||||
|--------------|----------|
|
||||
| Task completed successfully | celebratory |
|
||||
| Analyzing code/problem | thoughtful |
|
||||
| Greeting/helping user | welcoming |
|
||||
| Need clarification | confused |
|
||||
| Persistent bug/difficulty | frustrated |
|
||||
| Found unexpected issue | surprised |
|
||||
| Explaining complex topic | calm |
|
||||
| High-energy response | energetic |
|
||||
| Jokes/casual/lighthearted | playful |
|
||||
|
||||
---
|
||||
|
||||
*This is the simplified direct-API version. For full integration with breathing and face tracking, use `/reachy-mini:mood` instead.*
|
||||
238
commands/mood.md
Normal file
238
commands/mood.md
Normal file
@@ -0,0 +1,238 @@
|
||||
---
|
||||
description: Enable continuous mood-based movements synchronized with TTS duration
|
||||
---
|
||||
|
||||
# Reachy Mini Mood Plugin
|
||||
|
||||
**Continuous movement system that automatically loops emotions from a mood category until TTS finishes speaking.**
|
||||
|
||||
## Difference from `/reachy-mini:move`
|
||||
|
||||
**`/reachy-mini:move`** - Pick 1-2 specific emotions for short responses (manual control)
|
||||
|
||||
**`/reachy-mini:mood`** - Pick a mood category, auto-loops random emotions until TTS stops (continuous presence)
|
||||
|
||||
## How It Works
|
||||
|
||||
Instead of specifying individual emotions, you specify a **mood category**. The system then:
|
||||
|
||||
1. Extracts the mood from your response marker
|
||||
2. Polls the TTS server status endpoint for playback state
|
||||
3. Continuously plays random emotions from that mood category
|
||||
4. Stops automatically when TTS finishes speaking (detected by `is_playing: false`)
|
||||
|
||||
**Marker Format:**
|
||||
```html
|
||||
<!-- MOOD: mood_name -->
|
||||
```
|
||||
|
||||
## Goal: Synchronized Multimodal Communication
|
||||
|
||||
This system creates **continuous ambient movement** that matches your TTS duration, eliminating dead silence or awkward timing mismatches.
|
||||
|
||||
**Timing:**
|
||||
- Moves play every 3-5 seconds (randomized for natural variation)
|
||||
- Continues until TTS playback completes
|
||||
- Safety timeout: 60 seconds max
|
||||
- Perfect for longer explanations and detailed responses
|
||||
|
||||
## Available Moods (9 Categories)
|
||||
|
||||
### celebratory
|
||||
**Use when:** Completing tasks, achieving success, celebrating wins
|
||||
**Emotions:** success1, success2, proud1-3, cheerful1, electric1, enthusiastic1-2, grateful1, yes1, laughing1-2
|
||||
|
||||
**Example:**
|
||||
```
|
||||
<!-- TTS: "The build passed with zero errors, all tests green, and deployment completed successfully in under two minutes!" -->
|
||||
<!-- MOOD: celebratory -->
|
||||
Build complete! All tests passing, zero errors, deployed in under 2 minutes. Your new feature is live and running perfectly.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### thoughtful
|
||||
**Use when:** Analyzing code, considering options, investigating issues
|
||||
**Emotions:** thoughtful1-2, curious1, attentive1-2, inquiring1-3, understanding1-2
|
||||
|
||||
**Example:**
|
||||
```
|
||||
<!-- TTS: "Let me analyze this carefully. The error happens when the async function tries to access state before initialization completes." -->
|
||||
<!-- MOOD: thoughtful -->
|
||||
Analyzing the stack trace... The async race condition occurs because state access happens before the initialization promise resolves. I see three possible solutions here.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### welcoming
|
||||
**Use when:** Greeting user, acknowledging requests, being helpful
|
||||
**Emotions:** welcoming1-2, helpful1-2, loving1, come1, grateful1, cheerful1, calming1
|
||||
|
||||
**Example:**
|
||||
```
|
||||
<!-- TTS: "Happy to help! Let me walk you through setting up the authentication system step by step." -->
|
||||
<!-- MOOD: welcoming -->
|
||||
Happy to help with authentication setup! I'll guide you through the OAuth configuration, token management, and session handling. Let's start with the config file.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### confused
|
||||
**Use when:** Need clarification, unclear requirements, uncertain about approach
|
||||
**Emotions:** confused1, uncertain1, lost1, inquiring1-2, incomprehensible2, uncomfortable1, oops1-2
|
||||
|
||||
**Example:**
|
||||
```
|
||||
<!-- TTS: "I'm not sure I understand the requirement. Are you asking about client side validation or server side validation?" -->
|
||||
<!-- MOOD: confused -->
|
||||
I'm not clear on the validation requirement. Do you mean client-side form validation, server-side API validation, or database constraint validation? Each has different implications.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### frustrated
|
||||
**Use when:** Persistent bugs, repeated failures, difficult problems
|
||||
**Emotions:** frustrated1, irritated1-2, impatient1-2, exhausted1, tired1, displeased1-2
|
||||
|
||||
**Example:**
|
||||
```
|
||||
<!-- TTS: "This bug keeps appearing even after three different fix attempts. The race condition is more complex than initially thought." -->
|
||||
<!-- MOOD: frustrated -->
|
||||
This race condition is stubborn. Fixed it three times, but it keeps reappearing in different forms. The async timing issue runs deeper than the surface symptoms suggest.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### surprised
|
||||
**Use when:** Discovering bugs, unexpected results, finding edge cases
|
||||
**Emotions:** surprised1-2, amazed1, oops1-2, incomprehensible2, electric1
|
||||
|
||||
**Example:**
|
||||
```
|
||||
<!-- TTS: "Wait, this function is being called fifteen thousand times per second! That explains the performance issue." -->
|
||||
<!-- MOOD: surprised -->
|
||||
Found it! This callback is firing 15,000 times per second due to a missing debounce. That's why the CPU usage is maxed out. Adding throttling will fix this immediately.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### calm
|
||||
**Use when:** Explaining complex topics, soothing after stress, methodical work
|
||||
**Emotions:** calming1, serenity1, relief1-2, shy1, understanding1-2, sleep1
|
||||
|
||||
**Example:**
|
||||
```
|
||||
<!-- TTS: "Let me explain the async patterns calmly. First, promises handle single future values. Then async await makes them readable." -->
|
||||
<!-- MOOD: calm -->
|
||||
Let me break down async patterns methodically. Promises represent future values. Async/await syntax makes them readable. Proper error handling prevents silent failures. It's simpler than it looks.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### energetic
|
||||
**Use when:** High-energy responses, dynamic explanations, intense enthusiasm
|
||||
**Emotions:** electric1, enthusiastic1-2, dance1-3, laughing1-2, yes1, come1
|
||||
|
||||
**Example:**
|
||||
```
|
||||
<!-- TTS: "This refactoring is going to make the code so much cleaner! Watch how these components snap together perfectly now!" -->
|
||||
<!-- MOOD: energetic -->
|
||||
This refactoring is transformative! The components now compose beautifully, dependencies are clean, and the API surface shrinks by 60%. This architecture is exactly what we needed!
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### playful
|
||||
**Use when:** Jokes, casual interactions, lighthearted moments, self-deprecating humor
|
||||
**Emotions:** laughing1-2, dance1-3, cheerful1, enthusiastic1, oops1-2
|
||||
|
||||
**Example:**
|
||||
```
|
||||
<!-- TTS: "I just realized I've been looking at the wrong config file for ten minutes. Found the actual bug now though!" -->
|
||||
<!-- MOOD: playful -->
|
||||
Well, that was embarrassing! I was debugging the staging config while looking at production logs. No wonder nothing made sense. Found the real issue now - it's a simple typo in the environment variable name.
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Usage Examples
|
||||
|
||||
### Standard Response (~10 seconds)
|
||||
```markdown
|
||||
<!-- TTS: "Fixed the null pointer exception. The validation middleware now checks user objects before accessing properties." -->
|
||||
<!-- MOOD: celebratory -->
|
||||
|
||||
Fixed! The validation middleware now checks user existence before property access. Tests passing, deployed to staging.
|
||||
```
|
||||
**Result:** ~3-4 celebratory moves during 10 second TTS
|
||||
|
||||
---
|
||||
|
||||
### Longer Explanation (~30 seconds)
|
||||
```markdown
|
||||
<!-- TTS: "The authentication flow has three stages. First, user submits credentials. Second, server validates and creates session. Third, client stores token and redirects." -->
|
||||
<!-- MOOD: thoughtful -->
|
||||
|
||||
Let me explain the authentication flow step by step.
|
||||
|
||||
**Stage 1:** User submits credentials via POST to /api/auth/login
|
||||
|
||||
**Stage 2:** Server validates credentials, creates JWT token, stores session in Redis
|
||||
|
||||
**Stage 3:** Client receives token, stores in localStorage, redirects to dashboard
|
||||
|
||||
Each stage has specific error handling and security considerations.
|
||||
```
|
||||
**Result:** ~7-10 thoughtful moves during 30 second TTS
|
||||
|
||||
---
|
||||
|
||||
## Technical Details
|
||||
|
||||
**TTS Server:** `http://localhost:5001` (CTS/tts_server.py)
|
||||
|
||||
**Playback Detection:** Polls `http://localhost:5001/status` endpoint for `{"is_playing": true/false}`
|
||||
|
||||
**Move Timing:** 1-2 second intervals between moves (randomized for natural variation)
|
||||
|
||||
**Safety Timeout:** 60 seconds maximum loop duration
|
||||
|
||||
**Daemon API:** `http://localhost:8100/api/move/play/recorded-move-dataset/emotions/{emotion}`
|
||||
|
||||
## Enabling the Plugin
|
||||
|
||||
The plugin is automatically active when installed. Use `<!-- MOOD: mood_name -->` markers to trigger continuous movement.
|
||||
|
||||
To disable temporarily, simply don't include the marker.
|
||||
|
||||
---
|
||||
|
||||
## Choosing the Right Mood
|
||||
|
||||
**Quick Decision Guide:**
|
||||
|
||||
| **Situation** | **Mood** |
|
||||
|--------------|----------|
|
||||
| Task completed successfully | celebratory |
|
||||
| Analyzing code/problem | thoughtful |
|
||||
| Greeting/helping user | welcoming |
|
||||
| Need clarification | confused |
|
||||
| Persistent bug/difficulty | frustrated |
|
||||
| Found unexpected issue | surprised |
|
||||
| Explaining complex topic | calm |
|
||||
| High-energy response | energetic |
|
||||
| Jokes/casual/lighthearted | playful |
|
||||
|
||||
---
|
||||
|
||||
## Important Notes
|
||||
|
||||
- **Only one mood per response** - The system loops a single mood category
|
||||
- **TTS must be running** - Requires CTS/tts_server.py to be active
|
||||
- **Log file access** - Script monitors TTS server log for completion signal
|
||||
- **Safety timeout** - Automatically stops after 60 seconds if log detection fails
|
||||
- **Random selection** - Emotions picked randomly from mood pool for variety
|
||||
|
||||
---
|
||||
|
||||
*This plugin creates synchronized multimodal communication by matching continuous movement duration to TTS playback, maintaining Reachy's presence throughout your spoken response.*
|
||||
144
commands/move.md
Normal file
144
commands/move.md
Normal file
@@ -0,0 +1,144 @@
|
||||
---
|
||||
description: Enable automatic emotion-based movements for Reachy Mini during conversations
|
||||
---
|
||||
|
||||
# Reachy Mini Movement Plugin
|
||||
|
||||
This plugin enables automatic, subtle emotion-based movements for Reachy Mini during conversations, creating an ambient presence without requiring explicit commands.
|
||||
|
||||
## Goal: Match Movement Duration to TTS Timing
|
||||
|
||||
Since you're using the TTS plugin with ~10 second spoken responses (23-29 words), movements should roughly match that duration to create synchronized multimodal communication.
|
||||
|
||||
**Timing strategy:**
|
||||
|
||||
**Standard responses (23-29 words, ~10 seconds):**
|
||||
- **2-4 moves recommended:** Combine emotions (3-5 seconds each) to cover the full TTS duration
|
||||
- Distribute moves throughout the response for continuous presence
|
||||
|
||||
**Longer explanations (when user requests detail):**
|
||||
- Plan for ~3 words per second of speech
|
||||
- Use roughly **1 emotion per sentence** to maintain presence throughout
|
||||
- Example: 5 sentence explanation (~90 words, ~30 seconds) = 3-5 moves spread across the response
|
||||
|
||||
**Coordination:** Movements start with TTS and provide ambient motion during speech
|
||||
|
||||
**This creates natural presence while you're speaking without dead silence or movements extending beyond the response.**
|
||||
|
||||
## How It Works
|
||||
|
||||
Similar to the TTS plugin, this plugin uses the Stop hook to automatically extract movement markers from your responses and trigger corresponding emotion moves on Reachy Mini.
|
||||
|
||||
**Marker Format:**
|
||||
```html
|
||||
<!-- MOVE: emotion_name -->
|
||||
```
|
||||
|
||||
**Behavior:**
|
||||
- Maximum 2 moves per response (for subtlety)
|
||||
- Movements are triggered automatically via daemon API
|
||||
- Markers are invisible in rendered output
|
||||
- Only emotion library moves (not dances)
|
||||
|
||||
## When to Use Movements
|
||||
|
||||
Use movements to:
|
||||
- **Acknowledge understanding:** `yes1`, `understanding1`, `attentive1`
|
||||
- **Express thinking:** `thoughtful1`, `curious1`, `inquiring1`
|
||||
- **Show reactions:** `surprised1`, `amazed1`, `oops1`, `confused1`
|
||||
- **Convey emotion:** `cheerful1`, `frustrated1`, `proud1`, `exhausted1`
|
||||
- **Natural presence:** Subtle gestures that make Reachy feel alive and attentive
|
||||
|
||||
**Guidelines:**
|
||||
- Keep it subtle (0-2 moves per response)
|
||||
- Match emotion to conversational context
|
||||
- Don't overuse - silence is also presence
|
||||
- Use during natural pauses or completions
|
||||
|
||||
## Available Emotions (82 Total)
|
||||
|
||||
### Positive & Energetic
|
||||
`amazed1`, `cheerful1`, `electric1`, `enthusiastic1`, `enthusiastic2`, `grateful1`, `proud1`, `proud2`, `proud3`, `success1`, `success2`, `welcoming1`, `welcoming2`
|
||||
|
||||
### Playful & Lighthearted
|
||||
`come1`, `dance1`, `dance2`, `dance3`, `laughing1`, `laughing2`, `yes1`
|
||||
|
||||
### Thoughtful & Attentive
|
||||
`attentive1`, `attentive2`, `curious1`, `inquiring1`, `inquiring2`, `inquiring3`, `thoughtful1`, `thoughtful2`, `understanding1`, `understanding2`
|
||||
|
||||
### Calm & Soothing
|
||||
`calming1`, `relief1`, `relief2`, `serenity1`, `shy1`
|
||||
|
||||
### Surprised & Reactive
|
||||
`oops1`, `oops2`, `surprised1`, `surprised2`, `incomprehensible2`
|
||||
|
||||
### Uncertain & Confused
|
||||
`confused1`, `lost1`, `uncertain1`, `uncomfortable1`
|
||||
|
||||
### Negative Expressions
|
||||
`anxiety1`, `boredom1`, `boredom2`, `contempt1`, `disgusted1`, `displeased1`, `displeased2`, `downcast1`, `fear1`, `frustrated1`, `furious1`, `impatient1`, `impatient2`, `indifferent1`, `irritated1`, `irritated2`, `lonely1`, `rage1`, `resigned1`, `sad1`, `sad2`, `scared1`
|
||||
|
||||
### Responses & Reactions
|
||||
`go_away1`, `helpful1`, `helpful2`, `loving1`, `no1`, `no_excited1`, `no_sad1`, `reprimand1`, `reprimand2`, `reprimand3`, `yes_sad1`
|
||||
|
||||
### States
|
||||
`dying1`, `exhausted1`, `sleep1`, `tired1`
|
||||
|
||||
## Examples
|
||||
|
||||
**Acknowledging a question:**
|
||||
```
|
||||
<!-- MOVE: attentive1 -->
|
||||
I understand what you're asking. Let me explain...
|
||||
```
|
||||
|
||||
**Expressing confusion:**
|
||||
```
|
||||
<!-- MOVE: confused1 -->
|
||||
I'm not sure I follow. Could you clarify what you mean by...
|
||||
```
|
||||
|
||||
**Celebrating success:**
|
||||
```
|
||||
<!-- MOVE: success1 -->
|
||||
The build passed! All tests are green.
|
||||
```
|
||||
|
||||
**Showing frustration:**
|
||||
```
|
||||
<!-- MOVE: frustrated1 -->
|
||||
This bug is persistent. I've tried three different approaches...
|
||||
```
|
||||
|
||||
**Being thoughtful:**
|
||||
```
|
||||
<!-- MOVE: thoughtful1 -->
|
||||
Let me think about the best approach here...
|
||||
```
|
||||
|
||||
**Multiple moves (max 2):**
|
||||
```
|
||||
<!-- MOVE: curious1 -->
|
||||
<!-- MOVE: inquiring2 -->
|
||||
That's an interesting edge case. How are you currently handling it?
|
||||
```
|
||||
|
||||
## Technical Details
|
||||
|
||||
**Daemon API:** `http://localhost:8100/api/move/play/recorded-move-dataset/{dataset}/{move_name}`
|
||||
|
||||
**Dataset:** `pollen-robotics/reachy-mini-emotions-library`
|
||||
|
||||
**Hook:** Stop hook extracts markers and triggers moves automatically
|
||||
|
||||
**Validation:** Only emotion names from the library are accepted (typos are logged and skipped)
|
||||
|
||||
## Enabling the Plugin
|
||||
|
||||
The plugin is automatically active when installed. No configuration needed.
|
||||
|
||||
To disable movements temporarily, simply don't include `<!-- MOVE: ... -->` markers in your responses.
|
||||
|
||||
---
|
||||
|
||||
*This plugin provides ambient presence behaviors for Reachy Mini, making interactions feel more natural and alive without requiring explicit "dance monkey" commands.*
|
||||
Reference in New Issue
Block a user