gh-vuer-ai-vuer-skill-marketplace-skills-docs-vuer-ai/docs/tutorials/physics/mujoco-wasm.md

# MuJoCo WASM Integration

## Overview

The MuJoCo component enables running physics simulations directly in the browser using WebAssembly technology. This allows for real-time physics simulation without requiring server-side computation.

## Key Components

### Required Libraries
- **Library**: `@vuer-ai/mujoco-ts`
- **Version**: `0.0.24`
- **Entry point**: `dist/index.umd.js`

### Asset Management
You need to supply a list of paths to relevant files via the `assets` attribute. This includes:
- XML configuration files
- 3D meshes (OBJ format)
- Textures (PNG format)

## Complete Example: Cassie Robot

```python
import asyncio
from vuer import Vuer
from vuer.schemas import Scene, Fog, MuJoCo, ContribLoader

app = Vuer()

# Define all assets needed for the simulation
CASSIE_ASSETS = [
    "/static/mujoco/cassie/scene.xml",
    "/static/mujoco/cassie/cassie.xml",
    "/static/mujoco/cassie/pelvis.obj",
    "/static/mujoco/cassie/left-hip.obj",
    "/static/mujoco/cassie/left-thigh.obj",
    "/static/mujoco/cassie/left-shin.obj",
    "/static/mujoco/cassie/left-foot.obj",
    "/static/mujoco/cassie/right-hip.obj",
    "/static/mujoco/cassie/right-thigh.obj",
    "/static/mujoco/cassie/right-shin.obj",
    "/static/mujoco/cassie/right-foot.obj",
    "/static/mujoco/cassie/texture.png",
]

@app.spawn(start=True)
async def main(session):
    # Load the MuJoCo library
    session.upsert @ ContribLoader(
        library="@vuer-ai/mujoco-ts",
        version="0.0.24",
        entry="dist/index.umd.js",
        key="mujoco-loader",
    )

    # Wait for library to load
    await asyncio.sleep(2.0)

    # Set up the scene with MuJoCo's default styling
    session.set @ Scene(
        Fog(
            color=0x2C3F57,  # MuJoCo default background
            near=10,
            far=20,
        ),

        # Initialize MuJoCo simulation
        MuJoCo(
            src="/static/mujoco/cassie/scene.xml",
            assets=CASSIE_ASSETS,
            key="cassie-sim",
        ),
    )

    # Keep session alive
    while True:
        await asyncio.sleep(1.0)

app.run()
```

## Implementation Workflow

### 1. Load the Contrib Library

```python
session.upsert @ ContribLoader(
    library="@vuer-ai/mujoco-ts",
    version="0.0.24",
    entry="dist/index.umd.js",
    key="mujoco-loader",
)
```

### 2. Configure the Scene

Set up fog effects and background styling to match MuJoCo's default appearance:

```python
Fog(
    color=0x2C3F57,  # MuJoCo's default gray-blue
    near=10,
    far=20,
)
```

### 3. Provide Asset Paths

Supply URLs to all necessary model files:

```python
assets = [
    "/static/scene.xml",      # Main scene file
    "/static/robot.xml",      # Robot description
    "/static/mesh1.obj",      # 3D meshes
    "/static/mesh2.obj",
    "/static/texture.png",    # Textures
]
```

### 4. Initialize MuJoCo Component

```python
MuJoCo(
    src="/static/scene.xml",  # Main XML file
    assets=assets,            # All required assets
    key="mujoco-sim",
)
```

## Event Handling

Listen for simulation updates:

```python
async def on_mujoco_frame(event, session):
    """Handle physics updates"""
    print("MuJoCo frame:", event.value)
    # Access simulation state
    # Apply control inputs
    # Update visualization

app.add_handler("ON_MUJOCO_FRAME", on_mujoco_frame)
```

## Timing Considerations

### Option 1: Sleep Delay
```python
session.upsert @ ContribLoader(...)
await asyncio.sleep(2.0)  # Wait for library to load
session.set @ MuJoCo(...)
```

### Option 2: Event Listener
```python
async def on_contrib_load(event, session):
    """Initialize MuJoCo after library loads"""
    session.set @ MuJoCo(
        src="/static/scene.xml",
        assets=ASSETS,
    )

app.add_handler("ON_CONTRIB_LOAD", on_contrib_load)
```

## Asset Organization

Organize your assets directory:

```
static/mujoco/
├── cassie/
│   ├── scene.xml          # Main scene file
│   ├── cassie.xml         # Robot configuration
│   ├── pelvis.obj         # Body meshes
│   ├── left-hip.obj
│   ├── left-thigh.obj
│   ├── ...
│   └── texture.png        # Textures
└── gripper/
    ├── scene.xml
    ├── ...
```

## Serving Assets

Configure Vuer to serve your assets:

```python
app = Vuer(static_root="assets")
```

Then reference assets with `/static/` prefix:
```python
src="/static/mujoco/cassie/scene.xml"
```

## Best Practices

1. **Load library first** - Always load ContribLoader before MuJoCo component
2. **List all assets** - Include every file referenced in XML
3. **Use relative paths** - XML files should reference meshes with relative paths
4. **Match MuJoCo styling** - Use fog and background colors for consistency
5. **Handle loading time** - Wait for library to load before initialization

## Troubleshooting

### Simulation not appearing
- Verify all assets are accessible
- Check ContribLoader has loaded (wait or use event)
- Ensure XML file is valid MuJoCo format

### Missing textures/meshes
- Confirm all assets are in the assets list
- Check file paths in XML files
- Verify static_root configuration

### Performance issues
- Consider simplifying the model
- Reduce mesh polygon counts
- Optimize texture sizes

## Source

Documentation: https://docs.vuer.ai/en/latest/tutorials/physics/mujoco_wasm.html