Skip to content

fix: default G1 camera to WebRTC with webcam fallback#1565

Open
kaiknower wants to merge 1 commit intodimensionalOS:mainfrom
kaiknower:g1-webrtc-default-webcam-fallback
Open

fix: default G1 camera to WebRTC with webcam fallback#1565
kaiknower wants to merge 1 commit intodimensionalOS:mainfrom
kaiknower:g1-webrtc-default-webcam-fallback

Conversation

@kaiknower
Copy link

Summary

This change makes the Unitree G1 primitive stack use the built-in WebRTC camera by default, while keeping the previous local webcam path available as a fallback.

Changes

  • publish color_image and camera_info directly from G1Connection
  • publish camera TF for the G1 WebRTC camera path
  • switch the G1 primitive blueprint to use the Unitree camera by default
  • keep the previous webcam-based path behind DIMOS_G1_CAMERA_SOURCE=webcam
  • make G1 detection use the active camera calibration instead of always assuming the old ZED/webcam config
  • remove duplicate G1 connection wiring from the basic real/sim blueprints

Why

On headless G1 setups, there may be no local /dev/video* device available for the old webcam path.
Using the robot's built-in WebRTC camera makes the default setup work out of the box, while still preserving the previous behavior for anyone who wants to keep using a local webcam.

Usage

Default behavior:

  • use the Unitree built-in camera over WebRTC

Fallback to the previous local webcam path:

export DIMOS_G1_CAMERA_SOURCE=webcam

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 15, 2026

Greptile Summary

This PR makes the G1 robot default to its built-in WebRTC camera instead of requiring a local webcam, with the previous webcam-based path preserved behind the DIMOS_G1_CAMERA_SOURCE=webcam environment variable.

  • G1Connection now publishes color_image from the WebRTC video stream and camera_info/TF from a 1Hz background thread, matching the existing G1SimConnection pattern
  • Primitive blueprint centralizes camera source selection with a three-way switch (simulation → g1_sim_connection, webcam → camera_module, default → g1_connection), and exports G1_CAMERA_INFO for downstream modules
  • Detection blueprint now uses the dynamic G1_CAMERA_INFO instead of the previously hardcoded zed.CameraInfo.SingleWebcam, so calibration matches the active camera
  • Basic real/sim blueprints are simplified by removing duplicate connection wiring that is now handled inside the primitive blueprint
  • Note: G1_CAMERA_INFO doesn't account for simulation mode (MuJoCo uses different intrinsics than the WebRTC camera params) — a pre-existing issue that this PR has an opportunity to fix

Confidence Score: 4/5

  • This PR is safe to merge — it improves default hardware compatibility without breaking existing functionality.
  • The changes follow established patterns (G1SimConnection, Go2Connection), correctly centralize camera wiring, and preserve backward compatibility via environment variable. The simulation camera info mismatch is pre-existing and non-blocking. No new security concerns, no breaking API changes.
  • dimos/robot/unitree/g1/blueprints/primitive/uintree_g1_primitive_no_nav.py — the G1_CAMERA_INFO selection doesn't handle simulation mode correctly (pre-existing).

Important Files Changed

Filename Overview
dimos/robot/unitree/g1/connection.py Adds color_image and camera_info Out streams with a background thread publishing camera info and TF at 1Hz. Pattern mirrors existing G1SimConnection. Class-level camera_info_static is evaluated at import time, which is intentional and consistent with Go2/Mujoco patterns.
dimos/robot/unitree/g1/blueprints/primitive/uintree_g1_primitive_no_nav.py Adds three-way camera source selection (simulation/webcam/WebRTC) gated by env var. Exports G1_CAMERA_INFO for detection modules. The simulation branch uses G1Connection.camera_info_static (1280x720) which doesn't match MuJoCo's 320x240 camera — pre-existing issue, not a regression.
dimos/robot/unitree/g1/blueprints/perceptive/unitree_g1_detection.py Replaces hardcoded zed.CameraInfo.SingleWebcam with dynamic G1_CAMERA_INFO from the primitive blueprint. This correctly adapts detection calibration to match the active camera source.
dimos/robot/unitree/g1/blueprints/basic/unitree_g1_basic.py Removes duplicate g1_connection() wiring — connection is now handled inside the primitive blueprint. Clean simplification.
dimos/robot/unitree/g1/blueprints/basic/unitree_g1_basic_sim.py Removes duplicate g1_sim_connection() wiring — connection is now handled inside the primitive blueprint. Clean simplification.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["Module Import"] --> B{"global_config.simulation?"}
    B -->|Yes| C["_camera = g1_sim_connection()"]
    B -->|No| D{"DIMOS_G1_CAMERA_SOURCE"}
    D -->|"webcam"| E["_camera = camera_module(Webcam)"]
    D -->|"unitree (default)"| F["_camera = g1_connection()"]
    
    F --> G["G1Connection.start()"]
    G --> H["WebRTC video_stream → color_image"]
    G --> I["Thread: camera_info + TF @ 1Hz"]
    
    C --> J["G1SimConnection.start()"]
    J --> K["MuJoCo video_stream → color_image"]
    J --> L["Thread: camera_info @ 1Hz"]
    
    E --> M["CameraModule.start()"]
    M --> N["Webcam image_stream → color_image"]
    M --> O["rx.interval: camera_info @ 1Hz"]
    
    H & K & N --> P["Detection modules use G1_CAMERA_INFO"]
Loading

Last reviewed commit: 70dcdb6

Comment on lines +124 to +125
G1_CAMERA_SOURCE = _camera_source()
G1_CAMERA_INFO = G1Connection.camera_info_static if G1_CAMERA_SOURCE != "webcam" else zed.CameraInfo.SingleWebcam
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simulation uses mismatched camera intrinsics

When global_config.simulation is true (line 127), _camera correctly uses g1_sim_connection() which provides images from MuJoCo's 320x240 camera. However, G1_CAMERA_INFO is still set to G1Connection.camera_info_static (1280x720 from front_camera_720.yaml) because G1_CAMERA_SOURCE defaults to "unitree" regardless of simulation mode.

This means unitree_g1_detection.py will use 1280x720 camera intrinsics for 3D detection/tracking while receiving 320x240 images — leading to incorrect depth projections in simulation.

This was also wrong before this PR (it used zed.CameraInfo.SingleWebcam which was also wrong for sim), so this is pre-existing. But since this PR centralizes camera info selection, it's a good opportunity to fix it, e.g.:

G1_CAMERA_SOURCE = _camera_source()
if global_config.simulation:
    from dimos.robot.unitree.mujoco_connection import MujocoConnection
    G1_CAMERA_INFO = MujocoConnection.camera_info_static
elif G1_CAMERA_SOURCE != "webcam":
    G1_CAMERA_INFO = G1Connection.camera_info_static
else:
    G1_CAMERA_INFO = zed.CameraInfo.SingleWebcam

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant