Poor performance when sending large messages #

[Aismy]

When the message is less than 32KB, I grab the packet and find the rdma send operation used. At this time, the performance is better than tcp. However, when the message size is greater than 32KB, the rdma read operation used for packet capture and sending is much slower than tcp. why

[fruhland]

Are you working with the development branch? If not, you should definitely try that. In my experience, larger messages work just fine. You could try increasing buffer and slice lengths in Configuration.java. Maybe other values work better on your hardware.

[Aismy]

In addition, I found that the cpu usage will be extremely high. How can this be solved. High cpu in case of multiple connections

[Aismy]

Are you working with the development branch? If not, you should definitely try that. In my experience, larger messages work just fine. You could try increasing buffer and slice lengths in Configuration.java. Maybe other values work better on your hardware.

My test found that when the message reached about 30kb, rdma read operation was used, which seemed to be decided by ucx. This time will be particularly slow. I use ucx test also this problem. In addition, I found that the cpu usage is very high. Is there any way to solve this problem? I use it on netty. Thanks for the reply.

[fruhland]

By default, hadroNIO uses busy polling to achieve minimum latency. This comes at the cost of high CPU usage. However, since polling is only done during a select() call, it should not affect application performance negatively. Polling is not done randomly in the background, while your application is doing other stuff.
If you want to stress your CPU less, try setting POLL_METHOD in Configuration.java to DYNAMIC. This lets threads sleep, that did not receive any messages for some amount of time, but will increase latency. It does not make sense to use this in benchmarks, since the whole point of the benchmarks is to perform as many operations as possible, so there is no time to sleep anyways...