Skip to content

Stabilize the data unit tests#2169

Draft
jkamalu wants to merge 3 commits intoNVIDIA:mainfrom
jkamalu:data-unit-tests-update
Draft

Stabilize the data unit tests#2169
jkamalu wants to merge 3 commits intoNVIDIA:mainfrom
jkamalu:data-unit-tests-update

Conversation

@jkamalu
Copy link
Contributor

@jkamalu jkamalu commented Nov 7, 2025

This PR makes a few changes to address potential sources of instability/error in the data unit tests.

  1. The unit tests touched in this PR only need to be run on a single process, so in the multi-rank setting, we only do the work on one rank and return early on all others
  2. We revert to making an attempt to find test tokenizer vocab and merge files locally before downloading them from the web
  3. We fix the local path for the BERT vocab file
  4. We standardize somewhat the object storage client spoofing

@jkamalu jkamalu self-assigned this Nov 7, 2025
@jkamalu jkamalu requested a review from a team as a code owner November 7, 2025 04:47
@copy-pr-bot
Copy link

copy-pr-bot bot commented Nov 7, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@jkamalu jkamalu requested a review from ko3n1g November 7, 2025 04:48
@jkamalu jkamalu force-pushed the data-unit-tests-update branch 3 times, most recently from 1e7cb45 to e485838 Compare November 7, 2025 05:13
@jkamalu jkamalu added this to the Core 0.15 milestone Nov 7, 2025
@jkamalu
Copy link
Contributor Author

jkamalu commented Nov 7, 2025

/ok to test e485838

@jkamalu jkamalu force-pushed the data-unit-tests-update branch from e485838 to ea42e5f Compare November 7, 2025 17:06
@jkamalu
Copy link
Contributor Author

jkamalu commented Dec 3, 2025

/ok to test

@copy-pr-bot
Copy link

copy-pr-bot bot commented Dec 3, 2025

/ok to test

@jkamalu, there was an error processing your request: E1

See the following link for more information: https://docs.gha-runners.nvidia.com/cpr/e/1/

@jkamalu
Copy link
Contributor Author

jkamalu commented Dec 3, 2025

/ok to test d1f3558

@Phlip79
Copy link
Member

Phlip79 commented Mar 4, 2026

We are changing our review process and marking all open, unlabeled PRs as draft. This change will go in effect starting once #3659 is merged.

Moving forward, all PRs will be required to start as draft PRs. If you wish to get your PR merged, mark your PR as “Ready for review”. Read more about the new process at submit.md.

@Phlip79 Phlip79 marked this pull request as draft March 4, 2026 23:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants