Best practice in 2024 for Dask on AzureML Cluster?

[rsignell]

@james-tn , I'm working with a large research center that has AzureML and wants to run horizontally scaling Dask workflows. I'm hoping you can help me answer these questions:

1. What would you suggest to be the best way today to run Dask on an AzureML Cluster? Looks like dask-on-ray is not really supported anymore?

2. If there was funding to develop a package to run Dask clusters on AzureML Clusters more easily, what approach would you suggest?

[MaxSchambach]

Would be interested in this as well, not for Dask specifically but Ray. As this repositoy does not seem to be maintained anymore, what is the suggested solution to use dask or ray on AzureML @james-tn ?