Installation instructions appear outdated

[LemonWade]

Hi there,

When I try to run the following command

python diffusion_policy_code/sapien_env/sapien_env/teleop/pack_battery.py \
    --start_idx 0 \
    --end_idx 10 \
    --dataset_dir /data/tiga/work/CodeDiffuser/dataset \
    --resolution middle \
    --view default

I run into a number of problems:

1. Missing files

   • CodeDiffuser/diffusion_policy_code/general_dp/diffusion_policy/__init__.py
   • CodeDiffuser/diffusion_policy_code/general_dp/utils/__init__.py
   • CodeDiffuser/diffusion_policy_code/d3fields_dev/XMem/inference/data/mask_mapper.py

2. Extra steps for GroundingDINO
   The groundingdino package has to be installed manually from
   CodeDiffuser/diffusion_policy_code/GroundingDINO via

   pip install .

3. Missing Python dependencies
   PyMCubes, open3d, seaborn, trimesh, opencv‑python, transforms3d, segment‑anything, nlopt, openai, etc.

4. numpy version pin
   During installation numpy keeps being downgraded or pinned to 1.23.5.

5. Incomplete environment.yml
   The very last line of CodeDiffuser/environment.yml seems to be truncated or missing.

It looks like the Install section of the README is no longer in sync with the current codebase. Could you update the instructions (and possibly the requirements) so that others can reproduce your results more easily?

Thanks in advance!

[LemonWade]

I just hit another issue:

hydra.errors.InstantiationException: Error in call to target 'sapien_env.rl_env.pack_battery_env.PackBatteryRLEnv':
RuntimeError('Cannot find a suitable rendering device')

I’m going to try rerunning everything on a desktop Ubuntu 20.04 machine (with a display) and will share updates once I have more results.

[parkergu]

I’m going to try rerunning everything on a desktop Ubuntu 20.04 machine (with a display) and will share updates once I have more results.

Hi, did it run successfully?

[jia212]

@LemonWade Hello, I'm encountering the exact same issue as you regarding the "Cannot find a suitable rendering device" problem. My other project, which uses a higher version of Sapein, doesn't have this issue. I wonder if you can reproduce this project now. Thank you.

[LemonWade]

I’m very sorry that I haven’t checked my email messages for quite some time.

@parkergu There’s no issue running it on a PC with a display. You can replace the latter half of the environment.yaml file with the version I wrote:

  - s2n=1.0.10
  - s3transfer=0.6.2
  - scikit-image=0.19.3
  - pip:
    - opencv-contrib-python==4.9.0.80
    - matplotlib==3.8.2
    - dgl==1.1.3
    - open3d==0.18.0
    - scipy==1.12.0
    - PyYAML==6.0.1
    - lxml==5.1.0
    - PyMCubes==0.1.4
    - trimesh==4.0.0
    - git+https://github.com/facebookresearch/segment-anything.git
    - scikit-image==0.22.0
    - transforms3d==0.4.1
    - pyglet==1.5.27
    - gdown==4.7.1
    - git+https://github.com/WangYixuan12/GroundingDINO.git
    - pyrender
    - seaborn
    - h5py
    - progressbar2
    - gdown
    - gitpython
    - git+https://github.com/cheind/py-thin-plate-spline
    - hickle
    - tensorboard
    - numpy
    - sapien==2.2.2
    - pyperclip==1.8.2
    - git+https://github.com/ChaoningZhang/MobileSAM.git

When running the program, just install any missing packages as needed — but make sure that the numpy version remains fixed at 1.23.5.

[LemonWade]

I’m very sorry that I haven’t checked my email messages for quite some time.

@jia212 I’ve always kept the sapien version at 2.2.2, and the code was running fine last month.
If you find the original codebase a bit complex, I recommend following the CodeDiffuser approach, but using GroundingSAM2 and IDP3 as tools to reproduce the paper. That’s the strategy we used in practice. The main difficulty is extracting the point cloud of the dual-arm gripper, and for that part, I collaborated with someone who’s familiar with ROS.