eval: Isaac-lab Arena

molmo_spaces_isaac/README.md

@@ -44,7 +44,7 @@ uv pip install -e .[dev,sim]
 We have a helper script `ms-download` that can be used to grab the desired assets and
 scenes datasets in `usd` format, ready for use in `IsaacSim` and `IsaacLab`.
 
-- To get the assets for a specific dataset (e.g. `thor`, `objaverse`):
+- To get the assets for a specific dataset (e.g. `thor`):
 
 ```bash
 ms-download --type usd --install-dir assets/usd --assets thor
@@ -77,5 +77,44 @@ scene from the `ithor` dataset:
 <video src="https://github.com/user-attachments/assets/77fa8123-ec19-4ebf-95a8-5448c14ae826">
 </video>
 
-[0]: <https://docs.isaacsim.omniverse.nvidia.com/5.1.0/index.html> (isaacsim-website)
-[1]: <https://isaac-sim.github.io/IsaacLab/main/index.html> (isaaclab-website)
+---
+
+## Isaac Lab Arena (MolmoSpaces pick demo)
+
+Pick task only (THOR by default; add `--allow-objaverse` for Objaverse). Success = `lift_height >= succ_pos_threshold` (default 0.01 m).
+
+**Prerequisites:** Isaac Lab Arena installed from source ([Arena installation][3]). Set `ISAACLAB_ARENA_PATH` to the Arena repo root.
+
+**Assets:** Download THOR USDs and (optionally) iTHOR scene USDs into one root:
+
+```bash
+ms-download --type usd --install-dir /path/to/assets --assets thor --scenes ithor
+```
+
+**Run (single bundled episode):**
+
+```bash
+cd /path/to/IsaacLab-Arena
+./submodules/IsaacLab/isaaclab.sh -p /path/to/molmo_spaces_isaac/scripts/run_arena_benchmark_episode.py -- \
+  --assets_root /path/to/assets
+```
+
+`--scenes_root` defaults to `--assets_root` so iTHOR FloorPlans resolve automatically. Use `--scene_extra_xyz 0 0 Z` to tune vertical fit. Use `--benchmark_dir` + `--episode_idx N` to run a specific episode. List episodes without Isaac Sim: `python3 scripts/list_arena_benchmark_episodes.py --assets_root /path/to/assets`.
+
+**OpenPI / Pi0 (with cameras):** Cameras are auto-enabled with `--policy_type pi_remote`. Two DROID-style cameras: wrist (`panda_hand`) and exo shoulder (`panda_link0`, raised 1.56 m to match physical DROID geometry). Use `--joint_pos_policy` for joint-position pi0 models.
+
+```bash
+export ISAACLAB_ARENA_PATH=/path/to/IsaacLab-Arena
+export MOLMO_ISAAC_ASSETS_ROOT=/path/to/assets
+bash molmo_spaces_isaac/scripts/test_pi_remote.sh [episode_idx [steps]]
+```
+
+Camera obs keys: `camera_obs.wrist_cam_rgb`, `camera_obs.exo_cam_rgb` (override with `MOLMO_PI_WRIST_CAMERA` / `MOLMO_PI_EXO_CAMERA`). Enable cameras without pi_remote via `--with_cameras`.
+
+
+---
+
+[0]: https://docs.isaacsim.omniverse.nvidia.com/5.1.0/index.html
+[1]: https://isaac-sim.github.io/IsaacLab/main/index.html
+[2]: https://isaac-sim.github.io/IsaacLab-Arena/main/index.html
+[3]: https://isaac-sim.github.io/IsaacLab-Arena/main/pages/quickstart/installation.html

molmo_spaces_isaac/examples/benchmark_ithor_pick_hard_simple/README.md

@@ -0,0 +1,29 @@
+# benchmark_ithor_pick_hard_simple
+
+Simplified **THOR-only** pick benchmark for [Isaac Lab Arena](../../scripts/run_arena_benchmark_episode.py): same **iTHOR `house_index`** set and **robot placement** style as the published **FrankaPickHardBench** (first episode per house as template), but the pickup is always **`Bowl_27`** with a **heuristic** world pose (fixed offset in the robot frame: default **0.55 m forward, 0.42 m up** (tuned so the bowl clears the Arena Franka base and typical counter height), transformed by `robot_base_pose`).
+
+This is **not** MuJoCo pose parity: the bowl is not at the original sampled object pose. It is meant for **pipeline and visual** checks with **no Objaverse USDs**—only `objects/thor/` and iTHOR scene USDs under your install root.
+
+## Regenerate `benchmark.json`
+
+From the repo root (or anywhere), with `molmo_spaces_isaac` on `PYTHONPATH` or run from a venv where the package is installed:
+
+```bash
+python3 molmo_spaces_isaac/scripts/build_simple_ithor_thor_pick_benchmark.py \
+  --source /path/to/FrankaPickHardBench_.../benchmark.json \
+  --output molmo_spaces_isaac/examples/benchmark_ithor_pick_hard_simple/benchmark.json
+```
+
+Environment defaults for `--source`: `MOLMO_PICK_BENCHMARK_SOURCE` or `MOLMO_PICK_BENCHMARK_DIR` (file or directory containing `benchmark.json`).
+
+Options: `--max-houses K` (cap episode count), `--offset-forward`, `--offset-lateral`, `--offset-up` (meters in robot frame).
+
+## Assets
+
+Use a single **`ms-download`** install with **thor** assets and **ithor** scenes, e.g.:
+
+```bash
+ms-download --type usd --install-dir /path/to/assets --assets thor --scenes ithor
+```
+
+Pass **`--assets_root /path/to/assets`** to the Arena runner. For this bundled benchmark, **`--scenes_root` is optional**: if omitted, the runner defaults it to **`--assets_root`** (or `MOLMO_ISAAC_ASSETS_ROOT`) so one root is enough.