🤖 UniTeleop

A unified VR-based teleoperation framework for humanoid whole-body control.

🚀 Quick Start

# Create and activate conda environment
conda env create -f environment.yml
conda activate UniTeleop

🥳 UniTeleop-v1

A unified VR teleoperation system with dual-mode input (real-time/offline) and parallel control architecture for humanoid robots.

🎮 Usage

▶️ Real-time Mode

# 1. Edit config.json: "input_mode": "realtime"
# 2. Run system
python main.py
# 3. Connect VR device to http://[LAPTOP_IP]:8082/unity

📂 Offline Mode

# 1. Edit config.json: "input_mode": "offline"
# 2. Select recorded session (Optional)
python tools/data_manager.py --select
# 3. Run playback
python main.py

🎬 Record VR Data (Optional)

python tools/simple_recorder.py
# 's' = start, 'p' = stop, 'q' = quit

🍊 Other Examples

Demo1: Unified Whole Body Control

G1 robot unified policy control with web GUI and keyboard interfaces.

cd demo1-one-policy-unified-whole-body-control
# Web GUI control
python g1_8D_commands_control_web_GUI.py  # Web interface at http://localhost:5000
# Keyboard control  
python g1_8D_commands_control_keyboard.py # W/S/A/D movement, I/K/O/L torso control

Demo2: Offline Arm Control

VR-driven robot arm IK control with recorded VR data.

cd demo2-offline-arm-control
python offline_arm_only.py  # Load VR hand data for arm positioning

Demo3: Realtime Hand Retargeting

VR hand tracking to robot finger control with live camera feed.

cd demo3-realtime-hand-retargeting-camera-track
python main.py  # Real-time hand gesture → robot fingers

Demo4: Torso Control

VR head orientation to robot torso control with video streaming.

cd demo4-torso-control
python torso_control_realtime.py  # Head movement → torso posture

Demo5: PPO Motion Mimic

Reinforcement learning trained motion imitation with loco-mujoco.

Setup Environment

cd third-party/
git clone https://github.com/robfiras/loco-mujoco.git
cd loco-mujoco
cp ../../assets/LOCOMUJOCO_VARIABLES.yaml ./loco_mujoco/
pip install -e . 
pip install -e '.[smpl]'

Run Motion Demonstrations

cd examples/demo5-motion-mimic/
# Forward jump motion
python eval.py --path ../../assets/motion-mimic/PPOJax_saved_13_11_poses_forward_jump.pkl --use_mujoco
# Alternating squats
python eval.py --path ../../assets/motion-mimic/PPOJax_saved_22_14_poses_alternating_squats.pkl --use_mujoco
# G1 cartwheel motion
python eval_G1.py --path ../../assets/motion-mimic/PPOJax_saved-v5-cartwheel-parameter-18.pkl --use_mujoco

🌻 Acknowledgements

• TactAR: Meta VR App based on TactAR secondary development, thanks to Han Xue and Dakarai Crowder for discussions
• PHC: This project uses retargeting pipeline from PHC
• loco-mujoco: Motion mimic learning framework
• mujoco_playground: G1 robot locomotion policy training base code
• dex-retargeting: Hand motion retargeting base code
• mink: Inverse kinematics solver for robot control