---

IsaacLab-MLX

IsaacLab-MLX is a public fork of Isaac Lab focused on making the Isaac Lab workflow runnable on Apple Silicon macOS by replacing the CUDA-first training/runtime slice with MLX + mac-sim.

The fork is not trying to recreate full Omniverse, RTX, or PhysX GPU parity on macOS. The practical goal is narrower and more useful: preserve the high-level Isaac Lab task and training ergonomics wherever possible, introduce clean backend seams for compute and simulation, and build a Mac-native path that can grow task by task.

Why This Fork Exists

Upstream Isaac Lab is built around NVIDIA Isaac Sim and a CUDA-oriented runtime. That is the right choice for Linux and NVIDIA GPU deployments, but it leaves Apple Silicon users without a realistic local development path.

This fork exists to close that gap:

• keep Isaac Lab recognizable to existing users
• run the learning stack on Apple Silicon with MLX
• replace import-time CUDA and Omniverse crashes with explicit backend capability checks
• build a Mac-native simulator adapter for the first useful environments instead of waiting for full engine parity

Current Status

What works today:

• runtime/backend selection seam for torch-cuda|mlx and isaacsim|mac-sim
• lazy import boundaries in envs, sim, assets, sensors, managers, controllers, devices, scene, and markers so the macOS path fails explicitly instead of exploding on missing omni.*
• reproducible source bootstrap script for upstream repositories
• a runnable MLX + mac-sim cartpole vertical slice
• cartpole showcase variants covering Box, Discrete, MultiDiscrete, Tuple, and Dict spaces
• a runnable MLX + mac-sim cart-double-pendulum MARL slice with dict actions/observations/rewards
• a runnable MLX + mac-sim quadcopter slice with root-state dynamics
• MLX training, checkpoint save/load, and replay scripts
• portability guards for optional torch/warp utility imports on macOS
• smoke tests for the backend seam and mac-native task slices

What this does not claim yet:

• full Isaac Sim compatibility on macOS
• RTX sensors, camera pipelines, Warp kernels, cuRobo, or the full task zoo
• Isaac ROS acceleration or CUDA transport parity

MLX Port Architecture

The fork is organized around two explicit seams:

Compute backend

The compute backend isolates tensor/runtime concerns:

• device selection
• checkpoint save/load
• RNG and backend metadata
• room for MLX-native custom kernel paths

Current public options:

• torch-cuda: upstream Isaac Sim path
• mlx: Apple Silicon path

Implementation entrypoint:

• source/isaaclab/isaaclab/backends/runtime.py

Simulation backend

The simulation backend isolates the minimum simulator contract Isaac Lab environments need:

• reset(soft=...)
• step(render=..., update_fabric=...)
• joint state reads
• joint effort writes
• root/joint state writes
• backend capability reporting

Current public options:

• isaacsim: upstream runtime adapter
• mac-sim: Mac-native adapter path

The current mac-sim implementation is intentionally narrow. It currently covers cartpole, cart-double-pendulum, and quadcopter slices needed to prove the MLX training/inference path can run without CUDA.

Kernel backend

The kernel backend isolates Warp and future Metal custom-kernel paths:

• warp: upstream Isaac Sim + Warp path
• metal: Apple Silicon path for MLX + Metal-backed kernel replacements
• cpu: correctness fallback for bring-up and unsupported kernels

Today the public MLX tasks mostly use pure MLX ops. The explicit kernel selection seam exists now so future Metal kernels can land without rewriting task-facing APIs again.

MLX Quick Start

This path is for Apple Silicon macOS.

1. Create the environment with uv

cd IsaacLab
uv venv --python 3.11 .venv
uv pip install --python .venv/bin/python -e source/isaaclab[macos-mlx,dev]

If you want the upstream Isaac Sim runtime on Linux/NVIDIA instead, install the CUDA-side extra:

uv pip install --python .venv/bin/python -e source/isaaclab[cuda-isaacsim,dev]

2. Train the MLX cartpole baseline

PYTHONPATH=.:source/isaaclab .venv/bin/python \
  scripts/reinforcement_learning/mlx/train_cartpole.py \
  --num-envs 256 \
  --updates 200 \
  --rollout-steps 64 \
  --epochs-per-update 4 \
  --checkpoint logs/mlx/cartpole_policy.npz

3. Replay a trained checkpoint

PYTHONPATH=.:source/isaaclab .venv/bin/python \
  scripts/reinforcement_learning/mlx/play_cartpole.py \
  --checkpoint logs/mlx/cartpole_policy.npz \
  --episodes 3

Optional resume flow:

PYTHONPATH=.:source/isaaclab .venv/bin/python \
  scripts/reinforcement_learning/mlx/train_cartpole.py \
  --resume-from logs/mlx/cartpole_policy.npz \
  --checkpoint logs/mlx/cartpole_policy_resumed.npz \
  --updates 50

The torch-centric RL wrapper extension is now optional. If you need upstream SB3, rl-games, or RSL-RL wrappers, install them separately from the MLX path:

uv pip install --python .venv/bin/python -e source/isaaclab_rl[sb3]
uv pip install --python .venv/bin/python -e source/isaaclab_rl[rsl-rl]

For the MLX/mac-sim task slices documented in this fork, source/isaaclab_rl is not required.

4. Run cart-double-pendulum MARL smoke

PYTHONPATH=.:source/isaaclab .venv/bin/python \
  scripts/reinforcement_learning/mlx/play_cart_double_pendulum.py \
  --num-envs 64 --episodes 3 --max-steps 10000 --random-actions

5. Run quadcopter smoke

PYTHONPATH=.:source/isaaclab .venv/bin/python \
  scripts/reinforcement_learning/mlx/play_quadcopter.py \
  --num-envs 64 --episodes 3 --episode-length-s 0.5 --max-steps 10000 --thrust-action 0.2 --no-random-actions

6. Run the focused backend test suite

PYTHONPATH=.:source/isaaclab .venv/bin/pytest \
  scripts/tools/test/test_bootstrap_isaac_sources.py \
  source/isaaclab/test/backends/test_runtime.py \
  source/isaaclab/test/backends/test_portability_utils.py \
  source/isaaclab/test/backends/test_mac_cartpole.py \
  source/isaaclab/test/backends/test_mac_cartpole_showcase.py \
  source/isaaclab/test/backends/test_mac_cart_double_pendulum.py \
  source/isaaclab/test/backends/test_mac_quadcopter.py -q

Runtime Selection

The backend seam is exposed through the app/runtime layer:

• --compute-backend torch-cuda|mlx
• --kernel-backend warp|metal|cpu
• --sim-backend isaacsim|mac-sim

These flags are published through:

• source/isaaclab/isaaclab/app/app_launcher.py

Examples:

# Upstream path
python some_script.py --compute-backend torch-cuda --kernel-backend warp --sim-backend isaacsim

# macOS port path
python some_script.py --compute-backend mlx --kernel-backend metal --sim-backend mac-sim

Important constraint: AppLauncher now accepts --compute-backend mlx --sim-backend mac-sim in bootstrap mode, but it does not launch Isaac Sim/Omniverse there. Use the dedicated MLX scripts for task execution.

M5 Benchmarking

The fork now includes a dedicated MLX benchmark entrypoint for the current mac-native task set:

• scripts/benchmarks/mlx/benchmark_mac_tasks.py

Example:

PYTHONPATH=.:source/isaaclab .venv/bin/python \
  scripts/benchmarks/mlx/benchmark_mac_tasks.py \
  --tasks cartpole cart-double-pendulum quadcopter train-cartpole \
  --num-envs 256 \
  --steps 512 \
  --train-updates 20 \
  --json-out logs/benchmarks/mlx/m5-baseline.json

The benchmark emits:

• per-task env_steps_per_s for the current MLX/mac-sim env slices
• cartpole train_frames_per_s for the first MLX training loop
• runtime metadata including compute, kernel, sensor, and planner backend selection

Implemented MLX Vertical Slice

Current implemented slices:

• cartpole environment and trainer in source/isaaclab/isaaclab/backends/mac_sim/cartpole.py
• cartpole showcase space variants in source/isaaclab/isaaclab/backends/mac_sim/showcase.py
• cart-double-pendulum MARL environment in source/isaaclab/isaaclab/backends/mac_sim/cart_double_pendulum.py
• quadcopter environment in source/isaaclab/isaaclab/backends/mac_sim/quadcopter.py
• cartpole trainer entrypoint in scripts/reinforcement_learning/mlx/train_cartpole.py
• cartpole replay entrypoint in scripts/reinforcement_learning/mlx/play_cartpole.py
• cart-double-pendulum replay/smoke entrypoint in scripts/reinforcement_learning/mlx/play_cart_double_pendulum.py
• quadcopter replay/smoke entrypoint in scripts/reinforcement_learning/mlx/play_quadcopter.py

The cartpole path preserves the important upstream task semantics:

• observation order remains pole_pos, pole_vel, cart_pos, cart_vel
• reward structure matches upstream Isaac Lab cartpole
• termination conditions remain cart out-of-bounds or pole angle exceeding pi/2
• reset sampling keeps the pole-angle randomization behavior
• observations returned after done/reset are post-reset observations, matching the upstream direct RL flow
• cart-double-pendulum preserves per-agent dict observations/rewards/dones for cart and pendulum
• quadcopter preserves a root-state-centric policy observation layout with vectorized thrust/moment control

Bootstrapping Upstream Sources

This fork also includes a repository bootstrap script so the Isaac Lab / Isaac Sim / Isaac ROS sources can be cloned reproducibly into a workspace.

Script:

• scripts/bootstrap_isaac_sources.py

Example:

uv run scripts/bootstrap_isaac_sources.py \
  --dest ../isaac-sources \
  --with-isaacsim \
  --with-isaac-ros

The script:

• clones IsaacLab by default
• optionally clones IsaacSim
• optionally clones isaac_ros_common
• runs git lfs install / git lfs pull only for IsaacSim
• writes a manifest with remotes, requested refs, and resolved SHAs

What Has Changed Relative To Upstream

The most important fork changes are:

• backend/runtime abstractions in source/isaaclab/isaaclab/backends
• MLX cartpole reference implementation in source/isaaclab/isaaclab/backends/mac_sim
• lazy import handling in envs, sim, and utils so Mac users get explicit unsupported-backend errors
• CUDA device binding moved behind an adapter instead of direct unconditional torch.cuda.set_device(...)
• public README and bootstrap path oriented around a publishable MLX fork

Roadmap

Near-term priorities:

1. Expand mac-sim from the current task slices into a more general articulation/scene layer.
2. Add additional locomotion/manipulation tasks with compatible observation/action structure.
3. Push more task code through backend capability checks instead of import-time backend assumptions.
4. Define a stable checkpoint/config story across multiple MLX tasks.
5. Add Apple Silicon CI for the MLX backend slice.

Deferred work:

• Omniverse UI and Kit extension parity
• RTX camera and sensor parity
• Warp and CUDA custom kernel replacements beyond the first task set
• cuRobo and Isaac ROS CUDA transport features

Relationship To Upstream Isaac Lab

This repository is a fork of the upstream Isaac Lab project and still depends heavily on its architecture, APIs, tasks, and research workflow design.

Upstream remains the reference source for:

• task semantics
• asset conventions
• environment configuration patterns
• Isaac Sim integrations

Original upstream repository:

• isaac-sim/IsaacLab

Related reference repositories:

• isaac-sim/IsaacSim
• NVIDIA-ISAAC-ROS/isaac_ros_common
• ml-explore/mlx

Documentation

The upstream Isaac Lab documentation remains the best reference for the broader project model:

• Installation steps
• Reinforcement learning
• Tutorials
• Available environments

Isaac Sim Version Dependency

Upstream Isaac Lab is built on top of Isaac Sim and requires compatible Isaac Sim versions. This fork still treats Isaac Sim as the reference runtime for the upstream path.

Isaac Lab Version	Isaac Sim Version
main branch	Isaac Sim 4.5 / 5.0 / 5.1
v2.3.X	Isaac Sim 4.5 / 5.0 / 5.1
v2.2.X	Isaac Sim 4.5 / 5.0
v2.1.X	Isaac Sim 4.5
v2.0.X	Isaac Sim 4.5

Contributing

Contributions are welcome, especially in these areas:

• additional MLX-backed task ports
• generalized mac-sim asset and articulation support
• capability gating around CUDA-only integrations
• Apple Silicon verification and CI
• documentation for MLX and macOS users

For general upstream contribution guidance, see:

• Upstream contributing guide

Troubleshooting

Common current pitfalls:

• If mlx is missing, install it into the active uv environment.
• If imports fail on omni.* or isaacsim.*, you are probably invoking an upstream Isaac Sim path rather than the mac-sim path.
• If you use AppLauncher with --sim-backend mac-sim, it runs in bootstrap mode and returns a placeholder app object. Use the dedicated MLX scripts for environment stepping/training.
• If a task depends on cameras, Warp, cuRobo, or Omniverse-only features, it should currently be treated as unsupported on the macOS path.

Support

• Use GitHub Discussions for design discussions, roadmap ideas, and supported-task requests.
• Use GitHub Issues for concrete bugs, missing capability checks, broken MLX scripts, or backend portability problems.

License

The Isaac Lab framework is released under BSD-3 License. The isaaclab_mimic extension and its corresponding standalone scripts are released under Apache 2.0. The license files of dependencies and assets are present in the docs/licenses directory.

Isaac Lab still requires Isaac Sim for the upstream runtime path, and Isaac Sim includes components under proprietary licensing terms. Please see the Isaac Sim license for details.

The isaaclab_mimic extension requires cuRobo, which has proprietary licensing terms listed in docs/licenses/dependencies/cuRobo-license.txt.

Citation

If you use Isaac Lab or this MLX fork in your research, please cite the technical report:

@article{mittal2025isaaclab,
  title={Isaac Lab: A GPU-Accelerated Simulation Framework for Multi-Modal Robot Learning},
  author={Mayank Mittal and Pascal Roth and James Tigue and Antoine Richard and Octi Zhang and Peter Du and Antonio Serrano-Muñoz and Xinjie Yao and René Zurbrügg and Nikita Rudin and Lukasz Wawrzyniak and Milad Rakhsha and Alain Denzler and Eric Heiden and Ales Borovicka and Ossama Ahmed and Iretiayo Akinola and Abrar Anwar and Mark T. Carlson and Ji Yuan Feng and Animesh Garg and Renato Gasoto and Lionel Gulich and Yijie Guo and M. Gussert and Alex Hansen and Mihir Kulkarni and Chenran Li and Wei Liu and Viktor Makoviychuk and Grzegorz Malczyk and Hammad Mazhar and Masoud Moghani and Adithyavairavan Murali and Michael Noseworthy and Alexander Poddubny and Nathan Ratliff and Welf Rehberg and Clemens Schwarke and Ritvik Singh and James Latham Smith and Bingjie Tang and Ruchik Thaker and Matthew Trepte and Karl Van Wyk and Fangzhou Yu and Alex Millane and Vikram Ramasamy and Remo Steiner and Sangeeta Subramanian and Clemens Volk and CY Chen and Neel Jawale and Ashwin Varghese Kuruttukulam and Michael A. Lin and Ajay Mandlekar and Karsten Patzwaldt and John Welsh and Huihua Zhao and Fatima Anes and Jean-Francois Lafleche and Nicolas Moënne-Loccoz and Soowan Park and Rob Stepinski and Dirk Van Gelder and Chris Amevor and Jan Carius and Jumyung Chang and Anka He Chen and Pablo de Heras Ciechomski and Gilles Daviet and Mohammad Mohajerani and Julia von Muralt and Viktor Reutskyy and Michael Sauter and Simon Schirm and Eric L. Shi and Pierre Terdiman and Kenny Vilella and Tobias Widmer and Gordon Yeoman and Tiffany Chen and Sergey Grizan and Cathy Li and Lotus Li and Connor Smith and Rafael Wiltz and Kostas Alexis and Yan Chang and David Chu and Linxi "Jim" Fan and Farbod Farshidian and Ankur Handa and Spencer Huang and Marco Hutter and Yashraj Narang and Soha Pouya and Shiwei Sheng and Yuke Zhu and Miles Macklin and Adam Moravanszky and Philipp Reist and Yunrong Guo and David Hoeller and Gavriel State},
  journal={arXiv preprint arXiv:2511.04831},
  year={2025},
  url={https://arxiv.org/abs/2511.04831}
}

Acknowledgement

Isaac Lab development initiated from the Orbit framework. This fork builds on that work and on the upstream Isaac Lab architecture.