gh-vuer-ai-vuer-skill-marketplace-skills-docs-vuer-ai/docs/tutorials/camera/manipulating-camera-pose.md

Manipulating Camera Pose in Vuer

Overview

This tutorial demonstrates how to programmatically control virtual camera positions and orientations within the Vuer framework, along with tracking user interactions.

Key Concepts

CameraView Component

Virtual cameras in Vuer are controlled through the CameraView component with parameters:

• fov: Field of view in degrees
• width, height: Resolution in pixels
• position: Camera position [x, y, z]
• rotation: Camera rotation [x, y, z]
• matrix: 4x4 transformation matrix (alternative to position/rotation)
• stream: Streaming mode ("time", "frame", or "ondemand")
• fps: Frame rate for streaming
• near, far: Clipping planes

Complete Example

import asyncio
import pickle
from vuer import Vuer, VuerSession
from vuer.events import ClientEvent
from vuer.schemas import Scene, CameraView, DefaultScene, Urdf
from ml_logger import ML_Logger

# Initialize logger
logger = ML_Logger(root=".", prefix="assets")

# Load pre-recorded camera matrices
with open("assets/camera_movement.pkl", "rb") as f:
    data = pickle.load(f)
    matrices = [item["matrix"] for item in data]

app = Vuer()

# Event handler to track camera movements
async def track_movement(event: ClientEvent, sess: VuerSession):
    """Capture camera movement events"""
    if event.key != "ego":
        return

    logger.log(**event.value, flush=True, silent=True)
    print(f"Camera moved: {event.value['position']}")

app.add_handler("CAMERA_MOVE", track_movement)

@app.spawn(start=True)
async def main(proxy: VuerSession):
    # Set up the scene
    proxy.set @ Scene(
        DefaultScene(),

        # Add a robot for reference
        Urdf(
            src="/static/robot.urdf",
            position=[0, 0, 0],
            key="robot",
        ),
    )

    # Animate camera through recorded positions
    for i in range(len(matrices)):
        proxy.update @ [
            CameraView(
                key="ego",
                fov=50,
                width=320,
                height=240,
                matrix=matrices[i % len(matrices)],
                stream="time",
                fps=30,
                near=0.1,
                far=100,
            ),
        ]

        await asyncio.sleep(0.033)  # 30 FPS

    # Keep session alive
    while True:
        await asyncio.sleep(1.0)

app.run()

Dynamic Camera Control Methods

Method 1: Using Transformation Matrix

session.update @ CameraView(
    key="ego",
    matrix=[
        [1, 0, 0, x],
        [0, 1, 0, y],
        [0, 0, 1, z],
        [0, 0, 0, 1],
    ],
)

Method 2: Using Position and Rotation

session.update @ CameraView(
    key="ego",
    position=[x, y, z],
    rotation=[rx, ry, rz],  # Euler angles in radians
)

Method 3: Animated Camera Path

import math

for i in range(360):
    theta = math.radians(i)
    radius = 5

    # Circular orbit
    x = radius * math.cos(theta)
    z = radius * math.sin(theta)

    session.update @ CameraView(
        key="ego",
        position=[x, 2, z],
        rotation=[0, theta, 0],
    )

    await asyncio.sleep(0.033)  # 30 FPS

Replaying Recorded Movements

Load and replay pre-recorded camera movements:

import pickle

# Load recorded movements
with open("assets/camera_movement.pkl", "rb") as f:
    movements = pickle.load(f)

# Replay movements
for movement in movements:
    session.update @ CameraView(
        key="ego",
        matrix=movement["matrix"],
        fov=50,
        width=320,
        height=240,
    )

    await asyncio.sleep(0.033)  # 30 FPS

Event Handling

Track user-initiated camera movements:

async def track_movement(event: ClientEvent, sess: VuerSession):
    """Log user camera movements"""
    if event.key != "ego":
        return

    # Access camera data
    position = event.value.get("position")
    rotation = event.value.get("rotation")
    matrix = event.value.get("matrix")

    print(f"Position: {position}")
    print(f"Rotation: {rotation}")

    # Save to logger
    logger.log(**event.value, flush=True, silent=True)

app.add_handler("CAMERA_MOVE", track_movement)

Streaming Modes

"time" Mode

Continuous streaming at specified FPS:

CameraView(stream="time", fps=30)

"frame" Mode

Stream individual frames on demand.

"ondemand" Mode

Only render when explicitly requested (most efficient):

CameraView(stream="ondemand")

Best Practices

1. Use matrices for complex movements - More precise than position/rotation
2. Track user movements - Enable interactive camera control
3. Set appropriate FPS - Balance smoothness and performance
4. Use clipping planes - Optimize rendering with near/far settings
5. Use ondemand mode - Save resources when continuous streaming isn't needed

Source

Documentation: https://docs.vuer.ai/en/latest/tutorials/camera/move_camera.html