mlx5: Add support for TLP emulation#1706
Open
yishaih wants to merge 6 commits intolinux-rdma:masterfrom
Open
Conversation
To commit: 45bf337d0774 ("RDMA/mlx5: Add support for TLP VAR allocation").
Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
Extend mlx5dv_alloc_var() to support allocating TLP VAR regions used for Transaction Layer Packet emulation scenarios. Add MLX5DV_VAR_ALLOC_FLAG_TLP flag that applications pass to mlx5dv_alloc_var() to request a TLP VAR instead of a standard VAR. When this flag is set, the driver passes it to the kernel via MLX5_IB_ATTR_VAR_OBJ_ALLOC_FLAGS so the kernel allocates from the TLP VAR region. Signed-off-by: Maher Sanalla <msanalla@nvidia.com> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
Introduce mlx5dv_get_export_sizes() API that returns the buffer sizes required by the export/import APIs, that will be introduced in upcoming patches, to allow cross-process sharing of mlx5 objects (VAR, UMEM, DEVX object). Applications call this API to obtain the required buffer sizes, allocate accordingly, then pass the buffers to the per-object export/import functions. Signed-off-by: Maher Sanalla <msanalla@nvidia.com> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
Introduce mlx5dv_var_export(), mlx5dv_var_import() and mlx5dv_var_unimport() APIs to enable cross-process sharing of VAR objects. mlx5dv_var_export() exports the VAR into an opaque data buffer whose size is obtained via mlx5dv_get_export_sizes(). mlx5dv_var_import() creates a VAR object from the exported data on a context that originally allocated the VAR, or shares the same kernel resources (e.g., via ibv_import_device()). mlx5dv_var_unimport() releases the imported VAR without destroying the underlying kernel object. Signed-off-by: Maher Sanalla <msanalla@nvidia.com> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
Introduce mlx5dv_devx_obj_export(), mlx5dv_devx_obj_import() and mlx5dv_devx_obj_unimport() APIs to enable cross-process sharing of DEVX objects. mlx5dv_devx_obj_export() exports the DEVX object into an opaque data buffer whose size is obtained via mlx5dv_get_export_sizes(). mlx5dv_devx_obj_import() creates a DEVX object from the exported data on a context that originally created the object, or shares the same kernel resources (e.g., via ibv_import_device()). mlx5dv_devx_obj_unimport() releases the imported DEVX object without destroying the underlying kernel object. Signed-off-by: Maher Sanalla <msanalla@nvidia.com> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
Introduce mlx5dv_devx_umem_export(), mlx5dv_devx_umem_import() and mlx5dv_devx_umem_unimport() APIs to enable cross-process sharing of DEVX UMEM objects. mlx5dv_devx_umem_export() exports the DEVX UMEM into an opaque data buffer whose size is obtained via mlx5dv_get_export_sizes(). mlx5dv_devx_umem_import() creates a DEVX UMEM object from the exported data on a context that originally registered the UMEM, or shares the same kernel resources (e.g., via ibv_import_device()). mlx5dv_devx_umem_unimport() releases the imported DEVX UMEM without deregistering the underlying kernel object. Signed-off-by: Maher Sanalla <msanalla@nvidia.com> Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This series adds support for Transaction Layer Packet (TLP) emulation response gateway regions, enabling userspace device emulation software to write TLP responses directly to lower layers without kernel driver involvement.
Currently, the mlx5 driver exposes VirtIO emulation access regions via the MLX5_IB_METHOD_VAR_OBJ_ALLOC ioctl. This series extends that ioctl to also support allocating TLP response gateway channels for PCI device emulation use cases.
As part of that we introduce a few DV APIs to allow export/import of the involved objects (i.e. mlx5dv_devx_obj, mlx5dv_devx_umem, mlx5dv_var) between two processes which share the same underlay device context.
A detailed man pages exist as part of the series.
The matching kernel code was sent already to rdma-next