Document RAPIDS on Modal

[jacobtomlinson]

Investigate and document how to install RAPIDS and run a small example in Modal

[ncclementi]

I'm investigating how would this look like and I'm still a bit confused on the different way of running things and installing things. I'm going to write the questions here for documentation purposes and see if we can get Modal folks to shine some light on them.

If I understand correctly, the two different ways of running something in modal are via app/cli and via notebook, and

For app/cli :

• We can add packages to an existing image via pip https://modal.com/docs/guide/images#add-python-packages
  • I assume this will bring with it all the clobbering issues that pip might cause.
  • Unclear if this step needs to be done every time the app is run, my guess is yes.
  • Maybe there is a potential for cudf/cuml to be included in the base image
• We can provide our own image from a registry or via Dockerfile https://modal.com/docs/guide/existing-images
  • Here is a more advanced setup for inspiration https://modal.com/docs/guide/cuda#for-more-complex-setups-use-an-officially-supported-cuda-image

For notebook

• We can also install via pip in the notebook (same probably issues than above by installing via pip) https://modal.com/docs/guide/notebooks-modal#getting-started
• We can create our own image https://modal.com/docs/guide/notebooks-modal#creating-a-custom-image , although we need to deploy it first

We might want to document both cases and the possibilities in each of them. It would be great to confirm we have scoped this right.

[charlesfrye]

Exciting! Here's a quick snippet for installing cudf on Modal in App/CLI interface.

import modal

app = modal.App("install-cudf")

ctk_image = modal.Image.from_registry(
    "nvidia/cuda:12.9.0-runtime-ubuntu22.04", add_python="3.11"
).entrypoint([])  # removes chatty prints on entry

image = ctk_image.uv_pip_install(
    "cudf-cu12==25.8.*",
    extra_index_url="https://pypi.nvidia.com",
)


@app.function(image=image, gpu="H100")
def run():
    import cudf

    print("✅")

[ncclementi]

@charlesfrye It was great talking to you. This should get us started

• We should have a docs page soon for apps/functions
  • We will look into a more exciting run function besides the import.
• We will table for now notebooks until they are more mature (issue to track this separately Document RAPIDS on Modal notebooks #613)
• @jayavenkatesh19 is investigating why we were not able to install cuml when starting from the runtime.

Note:

• Explore if possible to do Zero Code Changes things via apps/functions.
  • This will most likely be via the import cudf, cudf.pandas.install() path at the top of the run() function, need to try it.

[jayavenkatesh19]

Think I've figured out why we were facing issues with the cuml installation.

By default, uv will only consider versions that are published on the first index that contains a given package, to avoid dependency confusion scenarios. So when we use extra_index_url="https://pypi.nvidia.com", a few dependencies of cuml that are not present on the pypi.nvidia.com but are present on pypi.org/simple are throwing up an version mismatch error. This is different from how pip handles multiple indexes.

The flag --index-strategy unsafe-best-match can be used to consider versions from all indexes. I ran into the exact same error message when installing cuml on the runtime Dockerfile using uv, but adding this flag to the command fixed the issue.

RUN uv pip install cuml-cu12==25.8.* --extra-index-url https://pypi.nvidia.com --index-strategy unsafe-best-match

This line installs cuml on the runtime image without any issues!

[ncclementi]

@charlesfrye I'm trying to pass --index-strategy unsafe-best-match astral docs here

But I get an error if I try to do this, since it's not an accepted arg for uv_pip_install()

image = ctk_image.uv_pip_install(
    "cudf-cu12==25.8.*", "cuml-cu12==25.8.*",
    extra_index_url="https://pypi.nvidia.com",
    index_strategy="unsafe-best-match"
)

hen I look in the docs https://modal.com/docs/reference/modal.Image#uv_pip_install I can't find a way to passing that.

It seems like it would be possible to add it here

https://github.com/modal-labs/modal-client/blob/f0462158142b9df76cf4d3fec6d97c691f605c5d/modal/image.py#L1288-L1297.

I think for now we can move to a simple cudf example, but ideally we would be able to install cuml.

[charlesfrye]

You can use the extra_options argument to pass in extra options like --index-strategy as a string, like

image = ctk_image.uv_pip_install(
    "cudf-cu12==25.8.*", "cuml-cu12==25.8.*",
    extra_options="--index-strategy unsafe-best-match",
    extra_index_url="https://pypi.nvidia.com",
)

That works to install cuml for me.

[ncclementi]

I'll give that a try, once I can get some things to run.

@charlesfrye I can sort off run pure cudf things. I'm running into some issues due to the the way python was compiled in the image. see astral-sh/python-build-standalone#683 (comment)

Separately, when I try a zero code change example, it hangs. I'm not getting any errors, nothing seems to happen. It gets stuck on the running step for quite long, until I decided to cut because it's clearly hanging. Any idea what could be happening? Logs son't seem to be exposing anything at this point.

This is the snippet.

import modal

app = modal.App("install-cudf")

ctk_image = modal.Image.from_registry(
    "nvidia/cuda:12.9.0-runtime-ubuntu22.04", add_python="3.12"
).entrypoint([])  # removes chatty prints on entry

image = ctk_image.uv_pip_install(
    "cudf-cu12==25.8.*", 
    extra_index_url="https://pypi.nvidia.com",
)


@app.function(image=image, gpu="A10")
def run():
    import cudf.pandas
    cudf.pandas.install()

    import pandas as pd
    import urllib.request
    import os
    
    # Download the file if it doesn't exist
    file_path = "/tmp/nyc_parking_violations_2022.parquet"
    if not os.path.exists(file_path):
        print("Downloading NYC parking violations dataset...")
        url = "https://data.rapids.ai/datasets/nyc_parking/nyc_parking_violations_2022.parquet"
        urllib.request.urlretrieve(url, file_path)
        print("Download data!")

    import time
    start_time = time.time()
    df = pd.read_parquet(
    "/tmp/nyc_parking_violations_2022.parquet",
    columns=["Registration State", "Violation Description", "Vehicle Body Type", "Issue Date", "Summons Number"]
    )

    (df[["Registration State", "Violation Description"]]
    .value_counts()
    .groupby("Registration State")
    .head(1)
    .sort_index()
    .reset_index()
    )
    end_time = time.time()  

    print(f"Time taken: {end_time - start_time} seconds")
    print(type(type(df)))

[ncclementi]

Blocked on: modal-labs/modal-client#3619

[mwaskom]

Here's my suggestion for installing Python from the latest version of the astral distribution on top of the nvidia image:

import modal

app = modal.App("install-cudf")

standalone_python_url = (
    "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/"
    "cpython-3.12.11+20250918-x86_64_v3-unknown-linux-gnu-install_only.tar.gz"
)
install_python_commands = [
    "RUN apt update",
    "RUN apt install -y curl",
    f"RUN curl -L -o /tmp/python-standalone.tar.gz {standalone_python_url}",
    "RUN tar -xzf /tmp/python-standalone.tar.gz -C /tmp",
    "RUN cp -r /tmp/python/* /usr/local",
]

ctk_image = (
    modal.Image.from_registry(
        "nvidia/cuda:12.9.0-runtime-ubuntu22.04",
        setup_dockerfile_commands=install_python_commands,
    ).entrypoint([])  # removes chatty prints on entry
)

image = ctk_image.uv_pip_install(
    "cudf-cu12==25.8.*",
    extra_index_url="https://pypi.nvidia.com",
)

However I'm still seeing your repro hanging on the cudf.pandas.install() so I may be missing something about the root cause issue.

[ncclementi]

@mwaskom I'm following the code above and I can confirm that the right the standalone installation is not working. This is a better reproducer than the cudf.pandas.install() . ANd When I run it I can confirm that the wrong algorithm is being installed.

import modal

app = modal.App("test-numba-standalone-python")

standalone_python_url = (
    "https://github.com/astral-sh/python-build-standalone/releases/download/20250918/"
    "cpython-3.12.11+20250918-x86_64_v3-unknown-linux-gnu-install_only.tar.gz"
)
install_python_commands = [
    "RUN apt update",
    "RUN apt install -y curl",
    f"RUN curl -L -o /tmp/python-standalone.tar.gz {standalone_python_url}",
    "RUN tar -xzf /tmp/python-standalone.tar.gz -C /tmp",
    "RUN cp -r /tmp/python/* /usr/local",
]

ctk_image = (
    modal.Image.from_registry(
        "nvidia/cuda:12.9.0-runtime-ubuntu22.04",
        setup_dockerfile_commands=install_python_commands,
    ).entrypoint([])  # removes chatty prints on entry
)

image = ctk_image.uv_pip_install(
    "cudf-cu12==25.8.*",
    extra_index_url="https://pypi.nvidia.com",
)


@app.function(image=image, gpu="A10")
def run():

    from numba import cuda
    @cuda.jit
    def kernel():
        pass

    print(kernel[1,1]())

    import sys

    print("sys.version: ", sys.version)
    print("sys.hash_info: ", sys.hash_info)

The output after modal run is

/usr/local/lib/python3.12/site-packages/numba_cuda/numba/cuda/dispatcher.py:687: NumbaPerformanceWarning: Grid size 1 will likely result in GPU under-utilization due to low occupancy.
  warn(NumbaPerformanceWarning(msg))
/usr/local/lib/python3.12/site-packages/numba/cpython/old_hashing.py:477: UserWarning: FNV hashing is not implemented in Numba. See PEP 456 https://www.python.org/dev/peps/pep-0456/ for rationale over not using FNV. Numba will continue to work, but hashes for built in types will be computed using siphash24. This will permit e.g. dictionaries to continue to behave as expected, however anything relying on the value of the hash opposed to hash as a derived property is likely to not work as expected.
  warnings.warn(msg)
None
sys.version:  3.12.11 (main, Sep 18 2025, 19:48:13) [Clang 20.1.4 ]
sys.hash_info:  sys.hash_info(width=64, modulus=2305843009213693951, inf=314159, nan=0, imag=1000003, algorithm='fnv', hash_bits=64, seed_bits=128, cutoff=0)

Notice that, sys.hash_info still returns the algorithm='fnv' which the same issue that we have when doing add_python=3.12, I reproted here astral-sh/python-build-standalone#683 (comment)

[ncclementi]

I think this is an issue with the astral build, compiled with algorithm='fnv'

I posted a reproducer in the astral repo astral-sh/python-build-standalone#683 (comment)

[charlesfrye]

@ncclementi I took a look at the slow execution you reported here, separately from the hash algorithm issue.

I was able to reproduce the behavior. I attached Modal's live tracing/profiling tool to an execution that appeared stuck and saw that it was sitting in the memory allocation step:

Didn't root-cause the issue, but I was able to avoid getting stuck by switching to the managed or pool values for CUDF_PANDAS_RMM_MODE. Is that acceptable?

Here's an example that runs. Note the additional .env call to set the mode for rmm. I also added a Modal Volume to cache the Parquet file for faster loads.

from pathlib import Path

import modal

app = modal.App("cudf-demo")

ctk_image = modal.Image.from_registry(
    "nvidia/cuda:12.9.0-runtime-ubuntu22.04", add_python="3.12"
).entrypoint([])  # removes chatty prints on entry

image = ctk_image.uv_pip_install(
    "cudf-cu12==25.8.*",
    extra_index_url="https://pypi.nvidia.com",
).env({"CUDF_PANDAS_RMM_MODE": "managed"})

data_cache = {
    "/data": modal.Volume.from_name(
        "nyc-parking-violations-2022", create_if_missing=True
    )
}


@app.function(image=image, gpu="A10", volumes=data_cache)
def run():
    import cudf.pandas

    cudf.pandas.install()

    import pandas as pd
    import urllib.request
    import os

    # download the file to a distributed Volume
    file_path = "/data/nyc_parking_violations_2022.parquet"
    if not os.path.exists(file_path):
        print("Downloading NYC parking violations dataset...")
        url = "https://data.rapids.ai/datasets/nyc_parking/nyc_parking_violations_2022.parquet"
        urllib.request.urlretrieve(url, file_path)
        print("Done!")

    import time

    start_time = time.time()

    df = pd.read_parquet(
        file_path,
        columns=[
            "Registration State",
            "Violation Description",
            "Vehicle Body Type",
            "Issue Date",
            "Summons Number",
        ],
    )

    (
        df[["Registration State", "Violation Description"]]
        .value_counts()
        .groupby("Registration State")
        .head(1)
        .sort_index()
        .reset_index()
    )
    end_time = time.time()

    print(f"Time taken: {end_time - start_time} seconds")
    print(type(type(df)))