Will zero-copy read/write (splicing) supported?

[zhangjaycee]

From my understanding, it seems that splicing is not supported yet, will this be supported? Thanks!

[jiangliu]

Splice() requires that one of the two fds is pipe at lease. The fusedev and virtiofs transport layer can't satisfy the requirement yet.
Recently there are some enhancements to the fuse protocol, which may help to do better than splice:)

[zhangjaycee]

Splice() requires that one of the two fds is pipe at lease. The fusedev and virtiofs transport layer can't satisfy the requirement yet.
Recently there are some enhancements to the fuse protocol, which may help to do better than splice:)

Sounds great! Would you provide some clues about what is to be introduced to the fuse protocol? Do you mean this one, passthrough read/write (https://www.spinics.net/lists/linux-fsdevel/msg180464.html)?

For me, I think it's cool if virtio-fs can support splicing, or fusedev can support something like DAX.

[jiangliu]

Splice() requires that one of the two fds is pipe at lease. The fusedev and virtiofs transport layer can't satisfy the requirement yet.
Recently there are some enhancements to the fuse protocol, which may help to do better than splice:)

Sounds great! Would you provide some clues about what is to be introduced to the fuse protocol? Do you mean this one, passthrough read/write (https://www.spinics.net/lists/linux-fsdevel/msg180464.html)?

For me, I think it's cool if virtio-fs can support splicing, or fusedev can support something like DAX.

yes, exactly!
And we are investigating io uring, which seems promise to optimize fuse-backend-rs performance:)

[liubogithub]

@jiangliu Perhaps the splice() @zhangjaycee mentioned is supposed to work between /dev/fuse and files in the passthrough directory?

[bsbernd]

@jiangliu Out of interest, did you look into io_uring? From my limited understand of it, it might not be compatible. What we need is an SQ, filled in by the kernel and consumed by userspace and then the other way around for CQ. But afaik, io_uring only supports userspace filling SQ, with the kernel consuming it. I will try to go through the design details and code during the next days.

I had ioctl threads with per core queues as in zufs and Miklos fuse2 branch on our todo list, but then figured out about rather high waitq wakeup latencies.https://www.spinics.net/lists/linux-fsdevel/msg214814.html
I experimental patches to allow kernel spinning and to reduce unneeded waitq wake (will send the patches today or tomorrow), but latency is still not ideal.
The SQ/CQ approach might work better, at least if a user space consumer thread is already running.

[jiangliu]

@jiangliu Out of interest, did you look into io_uring? From my limited understand of it, it might not be compatible. What we need is an SQ, filled in by the kernel and consumed by userspace and then the other way around for CQ. But afaik, io_uring only supports userspace filling SQ, with the kernel consuming it. I will try to go through the design details and code during the next days.

I had ioctl threads with per core queues as in zufs and Miklos fuse2 branch on our todo list, but then figured out about rather high waitq wakeup latencies.https://www.spinics.net/lists/linux-fsdevel/msg214814.html I experimental patches to allow kernel spinning and to reduce unneeded waitq wake (will send the patches today or tomorrow), but latency is still not ideal. The SQ/CQ approach might work better, at least if a user space consumer thread is already running.

We have tried to enable io_uring for fuse-backend-rs but encountered some issues related to io-uring async io framework. And it's time to resume the async io support now:) It would be great to give a second try for splice when enabling async io:)