Skip to content

feat: add remote sim and isaac support #14

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
blankey1337 wants to merge 5 commits into
base: main
Choose a base branch
from
Open

feat: add remote sim and isaac support #14

blankey1337 wants to merge 5 commits into from

Conversation

@blankey1337
Copy link
Owner

@blankey1337 blankey1337 commented Dec 1, 2025 •
edited
Loading

spin_and_wave2.mov

1. Robot Model (URDF) Improvements

  • File: software/src/lerobot/robots/alohamini/alohamini.urdf
  • Changes:
  • Replaced primitive shapes (boxes/cylinders) with high-fidelity CAD meshes (.stl) for the gripper base, gripper fingers, and wrist flex link.
  • Corrected the orientation of the gripper fingers to rpy="0 0 0" (and 180° for the mirrored finger) to ensure they face each other correctly.
  • Updated collision geometry to remain simple primitives for simulation stability.
  • New Files: Added STL mesh files to software/src/lerobot/robots/alohamini/meshes/.

2. Isaac Sim Environment (isaac_alohamini_env.py)

File: software/examples/alohamini/isaac_sim/isaac_alohamini_env.py

  • Fixes:
  • Headless Rendering: Explicitly added a UsdLux.DistantLight to the scene. This ensures the scene is illuminated even when the default environment fails to load or lacks lights compatible with the Raster renderer (used in headless mode).
  • Debug Checks: Added a runtime check that prints a warning if any camera returns a completely black image (max pixel value == 0).
  • Renderer Settings: Reverted to default renderer settings (commented out RayTracedLighting) to ensure stability, relying on the new light source for visibility.

3. Teleoperation & Visualization (remote_teleop_rerun.py)

  • File: software/examples/alohamini/remote_teleop_rerun.py
  • New Feature: Created this script to combine keyboard teleoperation with real-time Rerun visualization.
  • Fixes:
  • Rerun Hierarchy: Updated the script to prefix all camera images with camera/ (e.g., camera/head_front). This fixes the "Unknown entity" errors in the Rerun viewer.
  • Cleanup: Added logic to explicitly clear old keys (e.g., observation.head_front) from the Rerun viewer to remove persistent black artifacts.

4. Documentation

  • File: docs/remote_simulation_workflow.md
  • Updates: Updated the workflow documentation to reflect the new remote_teleop_rerun.py script instead of the web dashboard.

5. Utilities

  • File: software/examples/debug/visualize_urdf.py
  • Minor: Added "Starting visualize..." print statements for better feedback.
  • File: software/examples/alohamini/evaluate_bi.py
  • Minor: Added path handling to allow running from any directory.

Todo:

  1. Remove hardcoded IPs - move to config / command params
  2. Fix orientation / origin point on gripper fingers (see video)
shaky.mov

@blankey1337
Improve AlohaMini URDF with realistic meshes
6e360a5 
- Updated URDF to use STL meshes for gripper base, fingers, and wrist flex components instead of primitive shapes.
- Added STL files to software/src/lerobot/robots/alohamini/meshes/.
- Updated remote_teleop_rerun.py to prefix camera observation keys with 'camera/' to fix Rerun visualization hierarchy errors.
@blankey1337
Fix AlohaMini gripper orientation and Rerun camera visualization
e9ce986 
- Corrected gripper finger orientation in URDF (removed incorrect -90 deg rotation) so they align properly with the new meshes.
- Updated remote_teleop_rerun.py to clear old camera keys (observation.head_top, etc.) from Rerun to remove black placeholder boxes.
- Added debug checks for black images in isaac_alohamini_env.py and a fallback DistantLight to ensure the scene is lit in headless mode.
@blankey1337
Improve URDF meshes and fix Rerun visualization
52b9bc9 
- Replaced primitive gripper geometry with accurate STL meshes in URDF.
- Fixed gripper finger orientation.
- Added remote_teleop_rerun.py with correct Rerun entity hierarchy (camera/*).
- Added fallback lighting and debug checks to isaac_alohamini_env.py to fix black images in headless mode.
- Updated documentation.
@blankey1337
Remove obsolete dashboard app
281cd6e
@blankey1337
Remove accidental files and update .gitignore
a763d90
@blankey1337 blankey1337 changed the title Feature/urdf rerun fixes feat: add remote sim and isaac support Dec 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@blankey1337