CLI pipe may hang on large input

[olofhagsand]

The cli pipe feature may hang if the input is very large, on the order of thousands of lines.
For example, in test/test_pipe.sh, if the input is ~2000 lines, set1 | cat hangs.
Other examples have been reported involving large grep output.

[limenda]

Appears, it stucks at

cligen/cligen_io.c

Line 293 in 68fa225

if (write(s, inbuf, inbuflen) < 0){

as soon as it reaches the socket buffer size limit, which on my system is, for example, 212992. Since /proc/sys/net/core/rmem_max(wmem_max) was showing 16777216, I set the socket's SO_SNDBUF/SO_RCVBUF in cligen_eval_pipe_pre to be 16 Mb and that helped. I would propose adding the buffer size as setting to the clixon configuration yaml model. If you're fine with that - let me prepare and open MR.

[olofhagsand]

Good catch, thanks.
Yes as a work-around such a MR could be useful.

However, I could think the a select poll of outgoing buffers would be a better fix, ie dont try to write if the socket buffers are full, but that would need some more debugging and probably some redesign of the event code for pipes.

[limenda]

Yes, I understand this is just WA. From what I can see - the issue is not with the large request, but rather with the large response. grep/cat/whatever actually does their job and even responds back, but since the main process is still busy with writing and does not start reading, - it stucks because of the full buffer (and causes the main process to stuck as well). So there should be some way to begin reading the response earlier than all the request was sent. As of now I can't think of the nice solution.
One more thing should not it be len <= 0 here?

cligen/cligen_read.c

Line 952 in 68fa225

if (len == 0)

I've got len = -1 when there was ECONNRESET. len = -1 indicates the read call completed with the error.