WIP: wheels CI: write constraints directly to PIP_CONSTRAINT by jameslamb · Pull Request #21639 · rapidsai/cudf

jameslamb · 2026-03-03T21:31:49Z

Description

Contributes to rapidsai/build-planning#256 / rapidsai/build-planning#257

rapidsai/gha-tools#247 updates rapids-generate-pip-constraints in the following ways:

generates output for RAPIDS_DEPENDENCIES="latest" (previously silently generated an empty list of constraints)
appends to the passed-in file if it already exists
always passes use_cuda_wheels=true

That new behavior will help with testing wheels against different CTK versions via constraints on the cuda-toolkit metapackage.

Notes for Reviewers

Skipping `cudf-pandas` test scripts

That script generates some constraints for one install:

cudf/ci/cudf_pandas_scripts/run_tests.sh

Lines 60 to 74 in 84fe9cd

    
           rapids-generate-pip-constraints test_python_cudf_pandas ./constraints.txt 
        
           # notes: 
        
           # 
        
           #   * echo to expand wildcard before adding `[test,cudf-pandas-tests]` requires for pip 
        
           #   * need to provide --constraint="${PIP_CONSTRAINT}" because that environment variable is 
        
           #     ignored if any other --constraint are passed via the CLI 
        
           # 
        
           python -m pip install \ 
        
               -v \ 
        
               --constraint ./constraints.txt \ 
        
               --constraint "${PIP_CONSTRAINT}" \ 
        
               "$(echo "${CUDF_WHEELHOUSE}"/cudf_"${RAPIDS_PY_CUDA_SUFFIX}"*.whl)[test,cudf-pandas-tests]" \ 
        
               "$(echo "${LIBCUDF_WHEELHOUSE}"/libcudf_"${RAPIDS_PY_CUDA_SUFFIX}"*.whl)" \ 
        
               "$(echo "${PYLIBCUDF_WHEELHOUSE}"/pylibcudf_"${RAPIDS_PY_CUDA_SUFFIX}"*.whl)"

But doesn't want them active in later installs that put different pandas versions into the environment, because they'd conflict:

cudf/ci/cudf_pandas_scripts/run_tests.sh

Lines 132 to 139 in 84fe9cd

    
           if version_lte "${RAPIDS_PY_VERSION}" "3.13"; then 
        
               for version in "${versions[@]}"; do 
        
                   echo "Installing pandas version: ${version}" 
        
                   # This loop tests cudf.pandas compatibility with older pandas-numpy versions, 
        
                   # requiring numpy<2. cupy>=14 dropped support for numpy<2, so we explicitly 
        
                   # downgrade cupy here to avoid an import failure when cupy tries 
        
                   # to load against the older numpy. 
        
                   python -m pip install "numpy>=1.23,<2.0a0" "pandas==${version}.*" "cupy-cuda${RAPIDS_CUDA_VERSION%%.*}x<14"

Checklist

I am familiar with the Contributing Guidelines.
New or existing tests cover these changes.
The documentation is up to date with these changes.

copy-pr-bot · 2026-03-03T21:31:53Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

…raints file there

…append

jameslamb · 2026-03-04T21:03:15Z

/ok to test

jameslamb · 2026-03-04T21:03:50Z

ci/test_wheel_cudf_polars.sh

+# TODO(jameslamb): revert before merging
+git clone --branch generate-pip-constraints \
+    https://github.com/rapidsai/gha-tools.git \
+    /tmp/gha-tools
+
+export PATH="/tmp/gha-tools/tools:${PATH}"
+


Suggested change

# TODO(jameslamb): revert before merging

git clone --branch generate-pip-constraints \

https://github.com/rapidsai/gha-tools.git \

/tmp/gha-tools

export PATH="/tmp/gha-tools/tools:${PATH}"

jameslamb · 2026-03-04T21:03:58Z

ci/test_wheel_dask_cudf.sh

+# TODO(jameslamb): revert before merging
+git clone --branch generate-pip-constraints \
+    https://github.com/rapidsai/gha-tools.git \
+    /tmp/gha-tools
+
+export PATH="/tmp/gha-tools/tools:${PATH}"
+


Suggested change

# TODO(jameslamb): revert before merging

git clone --branch generate-pip-constraints \

https://github.com/rapidsai/gha-tools.git \

/tmp/gha-tools

export PATH="/tmp/gha-tools/tools:${PATH}"

jameslamb · 2026-03-04T21:04:14Z

ci/test_wheel_cudf.sh

+# TODO(jameslamb): revert before merging
+git clone --branch generate-pip-constraints \
+    https://github.com/rapidsai/gha-tools.git \
+    /tmp/gha-tools
+
+export PATH="/tmp/gha-tools/tools:${PATH}"
+


Suggested change

# TODO(jameslamb): revert before merging

git clone --branch generate-pip-constraints \

https://github.com/rapidsai/gha-tools.git \

/tmp/gha-tools

export PATH="/tmp/gha-tools/tools:${PATH}"

Contributes to rapidsai/build-planning#256 `rapids-generate-pip-constraints` currently special-cases `RAPIDS_DEPENDENCIES="latest"` and skips generating constraints in that case. This will be helpful in rapidsai/build-planning#256, where we want to start constraining `cuda-toolkit` in wheels CI based on the CTK version in the CI image being used. ## Notes for Reviewers ### How I tested this Looked for projects using this ([GitHub search](https://github.com/search?q=org%3Arapidsai+language%3AShell+%22rapids-generate-pip-constraints%22+AND+NOT+is%3Aarchived+&type=code)) and tested in them. It's just a few: * [ ] cudf (rapidsai/cudf#21639) * [ ] cuml (rapidsai/cuml#7853) * [ ] dask-cuda (rapidsai/dask-cuda#1632) * [ ] nvforest (rapidsai/nvforest#62) * [ ] raft (rapidsai/raft#2971) * [ ] rmm (rapidsai/rmm#2270) On all of those, wheels CI jobs worked exactly as expected and without needing any code changes or `dependencies.yaml` updates... so this PR is safe to merge any time. ### Is this safe? It should be (see "How I tested this"). This is only used to add **constraints** (not requirements), so it shouldn't change our ability to catch problems like "forgot to declare a dependency" in CI. It WILL increase the risk of `[test]` extras being underspecified. For example, if `cuml[test]` has `scikit-learn>=1.3` and the constraints have `scikit-learn>=1.5`, we might never end up testing `scikit-learn>=1.3,<1.5` (unless it's explicitly accounted for in a `dependencies: "oldest"` block). The other risk here is that this creates friction because constraints passed to `--constraint` cannot contain extras. So e.g. if you want to depend on `xgboost[dask]`, that cannot be in any of the lists generated by `rapids-generate-pipe-constraints`. I think we can work around that though when we hit those cases. Overall, I think these are acceptable tradeoffs. Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Bradley Dice (https://github.com/bdice) URL: #247

jameslamb · 2026-03-11T15:58:59Z

Closing this.

This was for testing rapidsai/gha-tools#247, which has now been merged.

The changes here will make it into the repov via #21671

wheels CI: write constraints directly to PIP_CONSTRAINT

f6fcc93

jameslamb added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Mar 3, 2026

github-actions bot assigned jameslamb Mar 3, 2026

Merge branch 'main' into pip-constraints-append

7b3b87a

jameslamb mentioned this pull request Mar 3, 2026

rapids-generate-pip-constraints: stop special-casing 'latest' rapidsai/gha-tools#247

Merged

6 tasks

jameslamb and others added 4 commits March 4, 2026 09:38

Merge branch 'main' into pip-constraints-append

b953e49

cudf_pands_scripts/run_tests.sh is special, keep using a custom const…

72e43ad

…raints file there

Merge branch 'main' of github.com:rapidsai/cudf into pip-constraints-…

7f7b6a0

…append

test new behavior

2451dc4

jameslamb commented Mar 4, 2026

View reviewed changes

jameslamb closed this Mar 11, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: wheels CI: write constraints directly to PIP_CONSTRAINT#21639

WIP: wheels CI: write constraints directly to PIP_CONSTRAINT#21639
jameslamb wants to merge 6 commits intorapidsai:mainfrom
jameslamb:pip-constraints-append

jameslamb commented Mar 3, 2026 •

edited

Loading

Uh oh!

copy-pr-bot bot commented Mar 3, 2026

Uh oh!

jameslamb commented Mar 4, 2026

Uh oh!

jameslamb Mar 4, 2026

Uh oh!

jameslamb Mar 4, 2026

Uh oh!

jameslamb Mar 4, 2026

Uh oh!

jameslamb commented Mar 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

	rapids-generate-pip-constraints test_python_cudf_pandas ./constraints.txt

	# notes:
	#
	# * echo to expand wildcard before adding `[test,cudf-pandas-tests]` requires for pip
	# * need to provide --constraint="${PIP_CONSTRAINT}" because that environment variable is
	# ignored if any other --constraint are passed via the CLI
	#
	python -m pip install \
	-v \
	--constraint ./constraints.txt \
	--constraint "${PIP_CONSTRAINT}" \
	"$(echo "${CUDF_WHEELHOUSE}"/cudf_"${RAPIDS_PY_CUDA_SUFFIX}"*.whl)[test,cudf-pandas-tests]" \
	"$(echo "${LIBCUDF_WHEELHOUSE}"/libcudf_"${RAPIDS_PY_CUDA_SUFFIX}"*.whl)" \
	"$(echo "${PYLIBCUDF_WHEELHOUSE}"/pylibcudf_"${RAPIDS_PY_CUDA_SUFFIX}"*.whl)"

	if version_lte "${RAPIDS_PY_VERSION}" "3.13"; then
	for version in "${versions[@]}"; do
	echo "Installing pandas version: ${version}"
	# This loop tests cudf.pandas compatibility with older pandas-numpy versions,
	# requiring numpy<2. cupy>=14 dropped support for numpy<2, so we explicitly
	# downgrade cupy here to avoid an import failure when cupy tries
	# to load against the older numpy.
	python -m pip install "numpy>=1.23,<2.0a0" "pandas==${version}." "cupy-cuda${RAPIDS_CUDA_VERSION%%.}x<14"

Conversation

jameslamb commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Notes for Reviewers

Skipping cudf-pandas test scripts

Checklist

Uh oh!

copy-pr-bot bot commented Mar 3, 2026

Uh oh!

jameslamb commented Mar 4, 2026

Uh oh!

jameslamb Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

jameslamb Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

jameslamb Mar 4, 2026

Choose a reason for hiding this comment

Uh oh!

jameslamb commented Mar 11, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

jameslamb commented Mar 3, 2026 •

edited

Loading

Skipping `cudf-pandas` test scripts