Hello, this is a very interesting piece of work!

[anonymousA01]

Hello, this is a very interesting piece of work! I was wondering if you have deployed the original pi0.5 model on actual hardware and tested its table tennis performance using synchronous inference? I'm really curious to see how its effectiveness compares to the new method you proposed.

[Sakits]

Hi @anonymousA01 ,

Thank you so much for your interest in our work!

We actually found that synchronous inference is almost completely not workable for table tennis. Since the model cannot perceive the environment during the action execution phase, it suffers from high reaction latency, often missing the ball or not reacting at all, simply because it didn’t capture the frame where the ball was visible.

Asynchronous inference significantly reduces control latency by enabling the model to frequently perceive the environment, even during action execution. This frequent perception–action loop is what allows us to achieve real-time table tennis rallies. :)

Best,
Jiaming

[anonymousA01]

Hi @anonymousA01 ,

Thank you so much for your interest in our work!

We actually found that synchronous inference is almost completely not workable for table tennis. Since the model cannot perceive the environment during the action execution phase, it suffers from high reaction latency, often missing the ball or not reacting at all, simply because it didn’t capture the frame where the ball was visible.

Asynchronous inference significantly reduces control latency by enabling the model to frequently perceive the environment, even during action execution. This frequent perception–action loop is what allows us to achieve real-time table tennis rallies. :)

Best, Jiaming

Thank you very much for your reply. Additionally, I have another question: was the expert demonstration data for playing table tennis collected through teleoperation?

[Sakits]

Yes, the demonstration data was collected via teleoperation using a robotic arm, which made the process quite challenging. In fact, we believe the current table tennis performance could be improved if the demonstrations were collected using VR or other more intuitive interfaces. The current bottleneck lies more in the data collection process than in the model itself.