gh-vuer-ai-vuer-skill-marketplace-skills-docs-vuer-ai/docs/tutorials/physics/mujoco-wasm.md

MuJoCo WASM Integration

Overview

The MuJoCo component enables running physics simulations directly in the browser using WebAssembly technology. This allows for real-time physics simulation without requiring server-side computation.

Key Components

Required Libraries

• Library: @vuer-ai/mujoco-ts
• Version: 0.0.24
• Entry point: dist/index.umd.js

Asset Management

You need to supply a list of paths to relevant files via the assets attribute. This includes:

• XML configuration files
• 3D meshes (OBJ format)
• Textures (PNG format)

Complete Example: Cassie Robot

import asyncio
from vuer import Vuer
from vuer.schemas import Scene, Fog, MuJoCo, ContribLoader

app = Vuer()

# Define all assets needed for the simulation
CASSIE_ASSETS = [
    "/static/mujoco/cassie/scene.xml",
    "/static/mujoco/cassie/cassie.xml",
    "/static/mujoco/cassie/pelvis.obj",
    "/static/mujoco/cassie/left-hip.obj",
    "/static/mujoco/cassie/left-thigh.obj",
    "/static/mujoco/cassie/left-shin.obj",
    "/static/mujoco/cassie/left-foot.obj",
    "/static/mujoco/cassie/right-hip.obj",
    "/static/mujoco/cassie/right-thigh.obj",
    "/static/mujoco/cassie/right-shin.obj",
    "/static/mujoco/cassie/right-foot.obj",
    "/static/mujoco/cassie/texture.png",
]

@app.spawn(start=True)
async def main(session):
    # Load the MuJoCo library
    session.upsert @ ContribLoader(
        library="@vuer-ai/mujoco-ts",
        version="0.0.24",
        entry="dist/index.umd.js",
        key="mujoco-loader",
    )

    # Wait for library to load
    await asyncio.sleep(2.0)

    # Set up the scene with MuJoCo's default styling
    session.set @ Scene(
        Fog(
            color=0x2C3F57,  # MuJoCo default background
            near=10,
            far=20,
        ),

        # Initialize MuJoCo simulation
        MuJoCo(
            src="/static/mujoco/cassie/scene.xml",
            assets=CASSIE_ASSETS,
            key="cassie-sim",
        ),
    )

    # Keep session alive
    while True:
        await asyncio.sleep(1.0)

app.run()

Implementation Workflow

1. Load the Contrib Library

session.upsert @ ContribLoader(
    library="@vuer-ai/mujoco-ts",
    version="0.0.24",
    entry="dist/index.umd.js",
    key="mujoco-loader",
)

2. Configure the Scene

Set up fog effects and background styling to match MuJoCo's default appearance:

Fog(
    color=0x2C3F57,  # MuJoCo's default gray-blue
    near=10,
    far=20,
)

3. Provide Asset Paths

Supply URLs to all necessary model files:

assets = [
    "/static/scene.xml",      # Main scene file
    "/static/robot.xml",      # Robot description
    "/static/mesh1.obj",      # 3D meshes
    "/static/mesh2.obj",
    "/static/texture.png",    # Textures
]

4. Initialize MuJoCo Component

MuJoCo(
    src="/static/scene.xml",  # Main XML file
    assets=assets,            # All required assets
    key="mujoco-sim",
)

Event Handling

Listen for simulation updates:

async def on_mujoco_frame(event, session):
    """Handle physics updates"""
    print("MuJoCo frame:", event.value)
    # Access simulation state
    # Apply control inputs
    # Update visualization

app.add_handler("ON_MUJOCO_FRAME", on_mujoco_frame)

Timing Considerations

Option 1: Sleep Delay

session.upsert @ ContribLoader(...)
await asyncio.sleep(2.0)  # Wait for library to load
session.set @ MuJoCo(...)

Option 2: Event Listener

async def on_contrib_load(event, session):
    """Initialize MuJoCo after library loads"""
    session.set @ MuJoCo(
        src="/static/scene.xml",
        assets=ASSETS,
    )

app.add_handler("ON_CONTRIB_LOAD", on_contrib_load)

Asset Organization

Organize your assets directory:

static/mujoco/
├── cassie/
│   ├── scene.xml          # Main scene file
│   ├── cassie.xml         # Robot configuration
│   ├── pelvis.obj         # Body meshes
│   ├── left-hip.obj
│   ├── left-thigh.obj
│   ├── ...
│   └── texture.png        # Textures
└── gripper/
    ├── scene.xml
    ├── ...

Serving Assets

Configure Vuer to serve your assets:

app = Vuer(static_root="assets")

Then reference assets with /static/ prefix:

src="/static/mujoco/cassie/scene.xml"

Best Practices

1. Load library first - Always load ContribLoader before MuJoCo component
2. List all assets - Include every file referenced in XML
3. Use relative paths - XML files should reference meshes with relative paths
4. Match MuJoCo styling - Use fog and background colors for consistency
5. Handle loading time - Wait for library to load before initialization

Troubleshooting

Simulation not appearing

• Verify all assets are accessible
• Check ContribLoader has loaded (wait or use event)
• Ensure XML file is valid MuJoCo format

Missing textures/meshes

• Confirm all assets are in the assets list
• Check file paths in XML files
• Verify static_root configuration

Performance issues

• Consider simplifying the model
• Reduce mesh polygon counts
• Optimize texture sizes

Source

Documentation: https://docs.vuer.ai/en/latest/tutorials/physics/mujoco_wasm.html