How to run in real-world scenes?

[ZJU-PLP]

@eanswer

Hi, Jie:

Thanks for sharing so nice work. I have run this code successfully in your given simulation scenes. However, could you mind sharing the details or scripts to implement this project in real-world robotic manipulation(showed in your demo)?

[eanswer]

Thank you for your interests. The policy deployed in real robot is trained purely in simulation using the code in this repo. The trained policy is directly plugged into real hardware code.

…

On Sat, Jan 6, 2024 at 23:06 Linpeng Peng ***@***.***> wrote: @eanswer <https://github.com/eanswer> Hi, Jie: Thanks for sharing so nice work. I have run this code successfully in your given simulation scenes. However, could you mind sharing the details or scripts to implement this project in real-world robotic manipulation(showed in your demo)? image.png (view on web) <https://github.com/eanswer/TactileSimulation/assets/32757213/da62f330-ecca-4979-9d72-653f22edb821> — Reply to this email directly, view it on GitHub <#8>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADXWD7CZSJHLNUHMIX4CX7LYNJCPHAVCNFSM6AAAAABBQDWAM2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGA3DSMBRHAYTCNQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ZJU-PLP]

Thank you for your interests. The policy deployed in real robot is trained purely in simulation using the code in this repo. The trained policy is directly plugged into real hardware code.
…
On Sat, Jan 6, 2024 at 23:06 Linpeng Peng @.> wrote: @eanswer https://github.com/eanswer Hi, Jie: Thanks for sharing so nice work. I have run this code successfully in your given simulation scenes. However, could you mind sharing the details or scripts to implement this project in real-world robotic manipulation(showed in your demo)? image.png (view on web) https://github.com/eanswer/TactileSimulation/assets/32757213/da62f330-ecca-4979-9d72-653f22edb821 — Reply to this email directly, view it on GitHub <#8>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADXWD7CZSJHLNUHMIX4CX7LYNJCPHAVCNFSM6AAAAABBQDWAM2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGA3DSMBRHAYTCNQ . You are receiving this because you were mentioned.Message ID: @.>

Thanks for your reply. Could you mind offering the scripts in your real-world robotic grasping?

[ZJU-PLP]

Thank you for your interests. The policy deployed in real robot is trained purely in simulation using the code in this repo. The trained policy is directly plugged into real hardware code.
…
On Sat, Jan 6, 2024 at 23:06 Linpeng Peng @.> wrote: @eanswer https://github.com/eanswer Hi, Jie: Thanks for sharing so nice work. I have run this code successfully in your given simulation scenes. However, could you mind sharing the details or scripts to implement this project in real-world robotic manipulation(showed in your demo)? image.png (view on web) https://github.com/eanswer/TactileSimulation/assets/32757213/da62f330-ecca-4979-9d72-653f22edb821 — Reply to this email directly, view it on GitHub <#8>, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADXWD7CZSJHLNUHMIX4CX7LYNJCPHAVCNFSM6AAAAABBQDWAM2VHI2DSMVQWIX3LMV43ASLTON2WKOZSGA3DSMBRHAYTCNQ . You are receiving this because you were mentioned.Message ID: @.>

@eanswer Hi, Jie:
There are several questions when to reproduce your project in real world robotic manipulation. Would you mind sharing the guidelines to solve these problems?

1. Can GelSlim3.0 sensors be replaced with DIGIT tactile sensors this project?
2. In the part of 4.5 Zero-Shot Sim-to-Real: Tactile RL Insertion Task of your paper, you have used domain randomization technique for better sim-to-real performance. Whether there is script code to deal with tactile sensor parameters variations in real-world manipulation by domain randomization technique?
3. Would you mind providing the scale of 3-D printed testing objects in peg-insertion tasks(including the physical sizes of peg, hole and pedestal)?

[eanswer]

Thanks for your interest again.

1. We didn't test other sensors on real robot. Theoretically, the simulator should work for DIGIT, but it requires separate parameters calibrations for the specific sensor.

2. We did randomization for several parameters of tactile sensor in our environment code as shown here.

3. The object size are 1:1 match between simulation and real robot.

[ZJU-PLP]

Thanks for your interest again.

1. We didn't test other sensors on real robot. Theoretically, the simulator should work for DIGIT, but it requires separate parameters calibrations for the specific sensor.
2. We did randomization for several parameters of tactile sensor in our environment code as shown here.
3. The object size are 1:1 match between simulation and real robot.

@eanswer
ok, I see. Thanks for your nice reply. However, I still do not know the CAD or STL object size(cube peg and cube pedestal contained three holes) used in real-world robotic insertion just showed in below. Would you mind sharing the 3D printing scale or CAD/STL file?