Releases: rapidsai/kvikio
Releases · rapidsai/kvikio
v26.02.00
What's Changed
🚨 Breaking Changes
- Add thread pool as function parameters to C++ API by @kingcrimsontianyu in #876
- Default to static linking of libcudart by @bdice in #885
🚀 New Features
- Expose cuFile stream register API to Python by @kingcrimsontianyu in #893
🛠️ Improvements
- Use strict priority in CI conda tests by @bdice in #877
- Use strict priority in CI conda tests by @bdice in #881
- Remove alpha specs from non-RAPIDS dependencies by @bdice in #883
- Enable merge barriers by @KyleFromNVIDIA in #884
- Add per-block-device thread pools for multi-drive I/O parallelism by @kingcrimsontianyu in #880
- Add devcontainer fallback for C++ test location by @bdice in #887
- Remove deprecated config setters and getters from python API by @kingcrimsontianyu in #882
- Empty commit to trigger a build by @bdice in #889
- Use SPDX license identifiers in pyproject.toml, bump build dependency floors by @jameslamb in #890
- Add CUDA 13.1 support by @bdice in #888
- build and test against CUDA 13.1.0 by @jameslamb in #891
- Empty commit to trigger a build by @jameslamb in #897
- Use main shared-workflows branch by @jameslamb in #900
- wheel builds: react to changes in pip's handling of build constraints by @mmccarty in #902
- fix(build): build package on merge to
release/*branch by @gforsyth in #907
New Contributors
Full Changelog: v26.02.00a...v26.02.00
v25.12.00
What's Changed
🚨 Breaking Changes
- Remove ability to build kvikio without CUDA by @vyasr in #829
- Minimize transitive includes by moving NVTX-related code to the detail namespace by @kingcrimsontianyu in #851
- Implement simple username-based authentication for WebHDFS by @kingcrimsontianyu in #859
- Add opportunistic Direct I/O support to POSIX backend by @kingcrimsontianyu in #863
- Require CUDA 12.2+ by @jakirkham in #860
🐛 Bug Fixes
- Fix an out-of-bounds memory access in file open flags checking by @kingcrimsontianyu in #853
- Restore isort for Cython files by @KyleFromNVIDIA in #864
- refactored update-version.sh to handle new branching strategy by @rockhowse in #868
- Pin Cython pre-3.2.0 and PyTest pre-9 by @jakirkham in #869
📖 Documentation
🛠️ Improvements
- Simplify async validation condition by @vyasr in #842
- Update
RAPIDS_BRANCH, codify changes inupdate-version.shby @KyleFromNVIDIA in #843 - Add named parameters for S3 credentials to Python API by @TomAugspurger in #846
- Create utility class URL builder and URL encoder. Fix a tricky URL path issue by @kingcrimsontianyu in #844
- Make parameter name more consistent for S3 utility functions by @kingcrimsontianyu in #848
- Enable
sccache-distconnection pool by @trxcllnt in #852 - Use main in RAPIDS_BRANCH by @bdice in #854
- Use main branch by @bdice in #855
- Use ruff and ruff-format instead of flake8, black, isort by @KyleFromNVIDIA in #856
- Use main shared-workflows branch by @bdice in #857
- Use SPDX for all copyright headers by @KyleFromNVIDIA in #858
- Use
RAPIDS_BRANCHin cmake-format invocations that need rapids-cmake configs by @bdice in #862 - Disable benchmark builds by default, enable in CI by @PointKernel in #866
- Use
sccache-distbuild cluster for conda and wheel builds by @trxcllnt in #865
New Contributors
- @PointKernel made their first contribution in #866
- @rockhowse made their first contribution in #868
Full Changelog: v25.12.00a...v25.12.00
v25.10.00
🚨 Breaking Changes
- Support access to public S3 (#820) @kingcrimsontianyu
- Reduce duplication between compat manager and defaults (#816) @vyasr
- Devendor libnvcomp from libkvikio (#805) @bdice
- Remove Python nvCOMP bindings and Zarr 2 support (#798) @vuule
🐛 Bug Fixes
- Keep string alive until nvtxDomainResourceCreate (#832) @vyasr
- Fix a bug in public S3 inference (#831) @kingcrimsontianyu
- Fix CUDA 13 handling of libcufile on aarch64 (#827) @bdice
- Skip max_device_cache_size setter when BAR1 memory isn't present on the GPUs in the system (#814) @ahoyle-nvidia
- Fix an S3 parsing bug in the open function. Improve regex usage (#810) @kingcrimsontianyu
- Remove remaining nvcomp references (#801) @vyasr
- Revert "Set compiler versions in context (#755)" (#784) @vyasr
- Relax mmap read requirement. Improve error message. (#781) @kingcrimsontianyu
🚀 New Features
- Support access to public S3 (#820) @kingcrimsontianyu
- Upgrade to nvCOMP 5.0.0.6 (#800) @vuule
- Remove Python nvCOMP bindings and Zarr 2 support (#798) @vuule
- Support WebHDFS (2/2): Python binding (#791) @kingcrimsontianyu
- Support file size query for S3 presigned URL (#789) @kingcrimsontianyu
- Support WebHDFS (1/2): C++ implementation (#788) @kingcrimsontianyu
- Support file-backed mapping (2/n): host/device read Python binding (#742) @kingcrimsontianyu
🛠️ Improvements
- Empty commit to trigger a build (#840) @msarahan
- Configure repo for automatic release notes generation (#825) @AyodeAwe
- Use C++20 for KvikIO main library (#819) @kingcrimsontianyu
- Explicitly manage TLS/SSL CA paths for remote I/O (#817) @kingcrimsontianyu
- Reduce duplication between compat manager and defaults (#816) @vyasr
- Added KVIKIO_REMOTE_VERBOSE option (#815) @TomAugspurger
- Use branch-25.10 again (#812) @jameslamb
- Update rapids-dependency-file-generator (#809) @KyleFromNVIDIA
- Add a unified remote I/O interface that infers the endpoint type from URL (2/2): Python binding (#808) @kingcrimsontianyu
- Devendor libnvcomp from libkvikio (#805) @bdice
- Remove more nvcomp packaging for conda (#804) @bdice
- Build and test with CUDA 13.0.0 (#803) @jameslamb
- Optionally require zarr>=3.0.0 (#802) @TomAugspurger
- Use build cluster in devcontainers (#797) @trxcllnt
- Improve KvikIO Python binding performance by releasing GIL wherever deemed necessary (#796) @kingcrimsontianyu
- Update rapids_config to handle user defined branch name (#794) @robertmaynard
- Add a unified remote I/O interface that infers the endpoint type from URL (1/2): C++ implementation (#793) @kingcrimsontianyu
- Update rapids-build-backend to 0.4.0 (#790) @KyleFromNVIDIA
- Fix C++20 warning in the mmap test (#785) @kingcrimsontianyu
- Allow latest OS in devcontainers (#780) @bdice
- Update build infra to support new branching strategy (#776) @robertmaynard
- Use GCC 14 in conda builds. (#756) @vyasr
- Use C++20 standard (#749) @kingcrimsontianyu
[NIGHTLY] v25.12.00
🔗 Links
🚨 Breaking Changes
🐛 Bug Fixes
- Fix an out-of-bounds memory access in file open flags checking (#853) @kingcrimsontianyu
🛠️ Improvements
- Enable
sccache-distconnection pool (#852) @trxcllnt - Make parameter name more consistent for S3 utility functions (#848) @kingcrimsontianyu
- Add named parameters for S3 credentials to Python API (#846) @TomAugspurger
- Create utility class URL builder and URL encoder. Fix a tricky URL path issue (#844) @kingcrimsontianyu
- Update
RAPIDS_BRANCH, codify changes inupdate-version.sh(#843) @KyleFromNVIDIA - Simplify async validation condition (#842) @vyasr
- Remove ability to build kvikio without CUDA (#829) @vyasr
v25.08.00
🚨 Breaking Changes
- Remove CUDA 11 from dependencies.yaml (#745) @KyleFromNVIDIA
- stop uploading packages to downloads.rapids.ai (#734) @jameslamb
🐛 Bug Fixes
- Fix remote I/O file size overflow in Python binding (#766) @kingcrimsontianyu
- Fix a case sensitivity bug in S3 remote I/O (#765) @kingcrimsontianyu
- Use RAPIDS' CUDA major-minor version in Conda's
cuda_version(#758) @vyasr - Fix unit test UB resulting from incorrect initialization (#751) @kingcrimsontianyu
- Add an overload getenv_or that supports env var alias. Add new env var KVIKIO_NUM_THREADS. Fix UB. (#735) @kingcrimsontianyu
📖 Documentation
- Remove CUDA 11 mentions from docs (#769) @pentschev
- add docs on CI workflow inputs (#764) @jameslamb
🚀 New Features
- Add the utility function to clear page cache (#741) @kingcrimsontianyu
- Support file-backed mapping (1/n): host/device read C++ implementation (#740) @kingcrimsontianyu
🛠️ Improvements
- Use libnvcomp conda package (#774) @bdice
- fix(docker): use versioned
-latesttag for allrapidsaiimages (#771) @gforsyth - Remove CUDA 11 workarounds (#770) @pentschev
- Use CUDA 12.9 in Conda, Devcontainers, Spark, GHA, etc. (#762) @jakirkham
- refactor(shellcheck): enable for all files and fix remaining warnings (#760) @gforsyth
- Remove nvidia and dask channels (#759) @vyasr
- Set compiler versions in context (#755) @jakirkham
- Use variants to produce separate builds with and without cufile support (#754) @vyasr
- Only use cufile on x86 (#753) @vyasr
- refactor(rattler): remove explicit
libnumapin now that upstream feedstock is fixed (#752) @gforsyth - Remove CUDA 11 from dependencies.yaml (#745) @KyleFromNVIDIA
- Add the conda dependency libnuma necessary for GDS I/O (#744) @kingcrimsontianyu
- Remove CUDA 11 devcontainers and update CI scripts (#738) @bdice
- refactor(rattler): remove cuda11 options and general cleanup (#737) @gforsyth
- stop uploading packages to downloads.rapids.ai (#734) @jameslamb
- Forward-merge branch-25.06 into branch-25.08 (#722) @gforsyth
- resolve forward-merge from branch-25.06 to branch-25.08 (#710) @gforsyth
v25.06.00
🚨 Breaking Changes
- Fix the S3 endpoint constructor ambiguity (#700) @kingcrimsontianyu
- Add support for temporary AWS credentials to access S3 (#693) @joosthooz
🐛 Bug Fixes
- [pytest] disable no-data-collected warning (#746) @madsbk
- Revert "Provide API to detect C2C platform and add NVML shim" (#731) @vyasr
- Check whether memory resource/BAR1 are supported (#729) @pentschev
- Use
try...finally...to ensure buffer is released (#728) @jakirkham - Fix benchmark code on MIG (#703) @jakirkham
- Fix the S3 endpoint constructor ambiguity (#700) @kingcrimsontianyu
- zarr-python 3.0.7 compatibility (#699) @TomAugspurger
- Vendor RAPIDS.cmake (#695) @bdice
- Fix setting region in C++ constructor instead of via the env var (#691) @joosthooz
📖 Documentation
- Fix doc build error (#723) @kingcrimsontianyu
- Fix comment typo in buffer_register (#669) @JigaoLuo
🚀 New Features
- Provide API to detect C2C platform and add NVML shim (#721) @kingcrimsontianyu
- Deprecate nvCOMP python API (#714) @kingcrimsontianyu
- Add the feature to check page cache residency information for a file (#704) @kingcrimsontianyu
- Add support for temporary AWS credentials to access S3 (#693) @joosthooz
🛠️ Improvements
- use 'rapids-init-pip' in wheel CI, other CI changes (#725) @jameslamb
- Finish CUDA 12.9 migration and use branch-25.06 workflows (#719) @bdice
- Update to clang 20 (#718) @bdice
- Quote head_rev in conda recipes (#717) @bdice
- Build and test with CUDA 12.9.0 (#715) @bdice
- feat: add support for Python 3.13 (#708) @gforsyth
- refactor(wheels): vendor nvcomp as temporary workaround (#702) @gforsyth
- Download build artifacts from Github for CI (#698) @VenkateshJaya
- Improve unit tests; add a dedicated macro for Linux system calls (#697) @kingcrimsontianyu
- Remove usage of APIs outside the limited API (#692) @vyasr
- Port all conda recipes to
rattler-build(#678) @gforsyth - Make page size portable by querying it at runtime (#677) @kingcrimsontianyu
- chore(conda): enable strict channel priority for conda build (#674) @gforsyth
- Moving wheel builds to specified location and uploading build artifacts to Github (#672) @VenkateshJaya
- Improve NVTX annotation for functions (#671) @kingcrimsontianyu
- Use latest release of cmake-maven-plugin (#652) @bdice
v25.04.00
🚨 Breaking Changes
- Improve config handling in cpp and python (#644) @kingcrimsontianyu
- Adjust the way of determining FileHandle's compatibility mode for sync and async I/O to improve code readability (#608) @kingcrimsontianyu
🐛 Bug Fixes
- Fix compilation error under openSUSE Tumbleweed (#667) @mrjoe7
- Another fix to update-version (#663) @raydouglass
- Add pom.xml to update-version.sh (#662) @raydouglass
- Fix an RPATH-related bug for CMake install (#661) @kingcrimsontianyu
- Minor fixes on the file permission, host allocation flag, and worker thread initialization (#637) @kingcrimsontianyu
- Fix package versioning (#616) @vyasr
- Fix std::future status query (#596) @kingcrimsontianyu
📖 Documentation
🚀 New Features
- Add C++ benchmarks (part 1/n) (#659) @kingcrimsontianyu
- Enable tracing of thread pool tasks using NVTX (#630) @kingcrimsontianyu
🛠️ Improvements
- Add macros KVIKIO_EXPECT and KVIKIO_FAIL to improve exception handling (#653) @kingcrimsontianyu
- nvCOMP : don't use legacy API (#650) @madsbk
- Use conda-build instead of conda-mambabuild (#648) @bdice
- Remove
FindCUDAToolkit.cmakefrom.pre-commit-config.yaml(#647) @KyleFromNVIDIA - Support zarr-python 3.x (#646) @TomAugspurger
- Remove
FindCUDAToolkitbackport (#645) @KyleFromNVIDIA - Improve config handling in cpp and python (#644) @kingcrimsontianyu
- Added timeout to kvikio requests (#643) @TomAugspurger
- Consolidate more Conda solves in CI (#636) @KyleFromNVIDIA
- Require CMake 3.30.4 (#635) @robertmaynard
- Use rapids-generate-version for package versions (#628) @bdice
- Forward-merge branch-25.02 into branch-25.04 (#626) @bdice
- Use shared-workflows branch-25.04 (#623) @bdice
- Add
shellcheckto pre-commit and fix warnings (#621) @gforsyth - Add build_type input field for
test.yaml(#620) @gforsyth - add telemetry (#615) @msarahan
- Add RAII file wrappers to avoid resource leak (#614) @kingcrimsontianyu
- disallow fallback to Make in Python builds (#612) @jameslamb
- Add
verify-codeownershook (#611) @KyleFromNVIDIA - Forward-merge branch-25.02 to branch-25.04 (#610) @bdice
- Adjust the way of determining FileHandle's compatibility mode for sync and async I/O to improve code readability (#608) @kingcrimsontianyu
- Migrate to NVKS for amd64 CI runners (#605) @bdice
- Retry on HTTP 50x errors (#603) @TomAugspurger
- Add tutorials to showcase GDS for nifti and dicom images (#597) @yiheng-wang-nv
- Apply east const using clang-format. Fix header issues reported by clang-tidy (#594) @kingcrimsontianyu
v25.02.01
🚨 Breaking Changes
- Continue to make KvikIO a shared library by moving code from hpp to cpp (#581) @kingcrimsontianyu
🐛 Bug Fixes
- Set
POSITION_INDEPENDENT_CODEon libcurl (#625) @KyleFromNVIDIA - allow deselecting nvcomp wheels (#589) @jameslamb
- Fix the misplaced libcurl in CMake (#587) @kingcrimsontianyu
🚀 New Features
🛠️ Improvements
- Update to nvcomp 4.2.0.11 (#639) @bdice
- Use
rapids-pip-retryin CI jobs that might need retries (#619) @gforsyth - Use std::string (#617) @madsbk
- Revert CUDA 12.8 shared workflow branch changes (#609) @vyasr
- Build and test with CUDA 12.8.0 (#600) @bdice
- Separate more interface and definition. Add comments on std::future. Mark noexcept to compat mode-related functions (#588) @kingcrimsontianyu
- Use GCC 13 in CUDA 12 conda builds. (#586) @bdice
- Zarr: pin to version 2.* (#583) @madsbk
- Continue to make KvikIO a shared library by moving code from hpp to cpp (#581) @kingcrimsontianyu
- Check if nightlies have succeeded recently enough (#579) @vyasr
- Update cuda-python lower bounds to 12.6.2 / 11.8.5 (#573) @bdice
- Improve compatibility mode consistency (#567) @kingcrimsontianyu
- Require approval to run CI on draft PRs (#563) @bdice
- Add breaking change workflow trigger (#536) @AyodeAwe
- Add Initial Java Support for GDS to KvikIO (#396) @aslobodaNV
v24.12.00
🚨 Breaking Changes
- Use curl >=8.5.0 to align with conda-forge and avoid CVEs. (#574) @bdice
- cufile version (#565) @madsbk
- Add a new KvikIO compatibility mode "AUTO" (#547) @kingcrimsontianyu
- Build KvikIO as a shared library (#527) @madsbk
- Small improvements (#493) @kingcrimsontianyu
🐛 Bug Fixes
- Use curl >=8.5.0 to align with conda-forge and avoid CVEs. (#574) @bdice
- cufile version (#565) @madsbk
- Fix the pytest error for async io (#559) @kingcrimsontianyu
- fix library-loading issues in editable installs (#553) @jameslamb
- Backport
FindCUDAToolkitfrom CMake 3.31 (#550) @KyleFromNVIDIA - Fix exporting of include directories (#540) @KyleFromNVIDIA
- Moving details in file_handle.hpp to .cpp (#539) @madsbk
- Disallow cuda-python 12.6.1 and 11.8.4 (#537) @bdice
- Fix case of find_package call (#534) @vyasr
- CurlHandle: fix error message handling (#522) @madsbk
- Don't use macros for cuda driver functions (#516) @Jacobfaib
- Fix CUDA driver type stub definitions (#511) @Jacobfaib
- Stop tagging wheels as arch-agnostic (#507) @KyleFromNVIDIA
- Include <utility> since we use std::exchange (#505) @robertmaynard
- Disabling curl tests doesn't now disable kvikio cpp tests (#503) @robertmaynard
- Visibility of static class methods (#492) @madsbk
📖 Documentation
- to (#532) @RichardScottOZ
🚀 New Features
- Add a new KvikIO compatibility mode "AUTO" (#547) @kingcrimsontianyu
- Upgrade nvcomp to 4.1.0.6 (#525) @bdice
🛠️ Improvements
- prefer wheel-provided libkvikio.so, use RTLD_LOCAL (#551) @jameslamb
- enforce wheel size limits, README formatting in CI (#548) @jameslamb
- remove WheelHelpers.cmake (#545) @jameslamb
- Put a ceiling on cuda-python (#543) @jameslamb
- Replace FindcuFile with upstream FindCUDAToolkit support (#542) @KyleFromNVIDIA
- Build KvikIO as a shared library (#527) @madsbk
- print sccache stats in builds (#524) @jameslamb
- RemoteIO: use a pinned bounce buffer (#519) @madsbk
- Use registered strings for NVTX. Add more NVTX annotations. (#518) @kingcrimsontianyu
- Minor README fixes (#517) @Jacobfaib
- Python bindings to
cuFileDriverOpen()andcuFileDriverClose()(#514) @madsbk - Add CUDA libs in Python Conda, Consolidate Conda CI installs & use
rapids-dask-dependency(#513) @jakirkham - S3 benchmark: adding cudf-kvikio and cudf-fsspec (#509) @madsbk
- Use Cython's
arrayto backPy_ssize_t[::1](#504) @jakirkham - Mark all of
Array'snogilcdeffunctions asnoexcept(#502) @jakirkham - Simplify
_to_stringencoding of Pythonstrs (#498) @jakirkham - make conda installs in CI stricter (#495) @jameslamb
- Small improvements (#493) @kingcrimsontianyu
- Prune workflows based on changed files (#489) @KyleFromNVIDIA
- Remote IO: S3 support (#479) @madsbk
- Use nvcomp wheel instead of bundling nvcomp (#478) @KyleFromNVIDIA
- Touch ups to
Array(#475) @jakirkham - Remote IO: http support (#464) @madsbk
v24.10.00
🚨 Breaking Changes
🐛 Bug Fixes
- Fix no-cuda build (#484) @madsbk
- Add license to kvikio wheel (#481) @raydouglass
- Fix release-after-main bug in
get_primary_cuda_context(#472) @madsbk - Intentionally leak static CUDA resources to avoid crash (part 2) (#462) @kingcrimsontianyu
- Fix several issues with wheel builds (#439) @KyleFromNVIDIA
- Fix missing 24.08 => 24.10 renames (#423) @madsbk
📖 Documentation
🚀 New Features
🛠️ Improvements
- nvtx support macos (#474) @madsbk
- Update update-version.sh to use packaging lib (#469) @AyodeAwe
- Remove now unnecessary variable (#466) @vyasr
- Use CI workflow branch 'branch-24.10' again (#465) @jameslamb
- Update to flake8 7.1.1. (#463) @bdice
- remove pytest dependency in kvikio conda recipe (#461) @jameslamb
- Add support for Python 3.12, relax numcodecs pin (#458) @jameslamb
- Skip cufile and nvtx3 when cuda isn't found (#455) @madsbk
- Non-static bounce buffer option (#454) @madsbk
- Use nvcomp defaults for algo options. (#450) @bdice
- Update rapidsai/pre-commit-hooks (#448) @KyleFromNVIDIA
- Make the bounce buffer size configurable (#447) @madsbk
- Update
build.sh(#445) @madsbk - Drop Python 3.9 support (#443) @jameslamb
- Remove NumPy <2 pin (#441) @seberg
- C++ Test (#440) @madsbk
config_path(): return a reference (#438) @madsbk- Stream sync before calling cufile (#437) @madsbk
- Speeding up Python tests (#435) @madsbk
- Cython refactoring (#434) @madsbk
- Update pre-commit hooks (#433) @KyleFromNVIDIA
- Improve update-version.sh (#432) @bdice
- Use tool.scikit-build.cmake.version, set scikit-build-core minimum-version (#430) @jameslamb
- build.sh: --pydevelop (#429) @madsbk