Skip to content

Conversation

@yhori991
Copy link

@yhori991 yhori991 commented Apr 29, 2025

Checklist

I've tested the build only for Linux-64 Python3.12 environment so far.
Let me see how it goes on all the other platforms.

@conda-forge-admin
Copy link
Contributor

conda-forge-admin commented Apr 29, 2025

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe/meta.yaml) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe/meta.yaml:

  • ℹ️ The recipe is not parsable by parser conda-souschef (grayskull). This parser is not currently used by conda-forge, but may be in the future. We are collecting information to see which recipes are compatible with grayskull.

This message was generated by GitHub Actions workflow run https://github.com/conda-forge/conda-forge-webservices/actions/runs/15041903025. Examine the logs at this URL for more detail.

@yhori991 yhori991 changed the title Fix #72 Fix #72: Add java artifacts Apr 29, 2025
@mattip
Copy link
Contributor

mattip commented Apr 30, 2025

Do the linux builds take too much memory and trigger an OOM error? They are crashing after ~5 1/2 hours with no error message.

@yhori991
Copy link
Author

yhori991 commented May 1, 2025

I think so too. The way Linux builds failed looked to me like the one from OOM.
For all cases, the log stream seemingly got cut off without any signs of logical errors.
I hope that we can tweak conda-forge CI/CD pipeline settings for this...

The windows build on the other hand looks to me simpler.
We don't have unzip package for windows platform and we can exclude windows on this.

However, the bezel build recipe is using unzip internally.
https://github.com/ray-project/ray/blob/ea987b5cfc6464ee74ace6ce81bbfed1792d3d93/java/BUILD.bazel#L286
So, in the end, it might be necessary to support unzip on windows to solve this.

@yhori991
Copy link
Author

yhori991 commented May 1, 2025

FWIW, it seems that the upstream does not support building/testing the Java artifacts on the Windows platform.
Ref: https://github.com/ray-project/ray/blob/a90a8184f990a18b031cf95cc2ecf44e0474f6fe/ci/ray_ci/windows/tests.env.Dockerfile#L14
Setting RAY_INSTALL_JAVA=0 in this file indicates that the Windows build does not include Java bindings.

@mattip
Copy link
Contributor

mattip commented May 1, 2025

What is the motivation for this PR?

I hope that we can tweak conda-forge CI/CD pipeline settings for this

I don't know how to use a larger machine in conda-forge/azure, maybe someone else in the @conda-forge/ray-packages team knows. I don't understand why it takes 5 hours to build ray. Locally the build takes ~20 minutes.

@yhori991
Copy link
Author

yhori991 commented May 1, 2025

Well... my main motivation is to run RayDP purely on conda environments.
It's already possible by mixing PyPi and installing ray and raydp from there but it's so hard to do that without breaking something.

I don't understand why it takes 5 hours to build ray. Locally the build takes ~20 minutes.

It takes roughly the same amount of time (~20mins) on my local machine too.

@apmorton
Copy link
Contributor

apmorton commented May 1, 2025

It takes forever because the azure devops runners have 2 cores.

I don't know how to use a larger machine in conda-forge/azure, maybe someone else in the @conda-forge/ray-packages team knows.

I've put a request in for larger runners for this feedstock - conda-forge/admin-requests#1509

@mattip
Copy link
Contributor

mattip commented May 11, 2025

Close/reopen to use the larger linux runnuers

@mattip mattip closed this May 11, 2025
@mattip mattip reopened this May 11, 2025
@mattip
Copy link
Contributor

mattip commented May 11, 2025

The security policy on the larger machines requires that the commit come from someone approved to use them, so I pushed an empty commit in my name.

@mattip mattip force-pushed the enhancement-include-java-bindings branch from a49edd5 to b9f26d1 Compare May 11, 2025 09:30
@mattip
Copy link
Contributor

mattip commented May 11, 2025

Empty commit only ran windows and macOS CI jobs, maybe rebasing is needed to get the linux ones to run?

@mattip
Copy link
Contributor

mattip commented May 11, 2025

@jaimergp any ideas why this is not running the linux CI after merging #210? Do I need to submit it from my fork?

@jaimergp
Copy link
Member

Not sure, maybe it has to do with the conflicts? IIRC, GHA won't run in pull_request triggers if there are conflicts. I'd try fixing those and if that doesn't fix the runs, re-open the PR from your fork? (But I really think it should work once conflicts are fixed).

@yhori991
Copy link
Author

I've fixed the conflicts. Let's see 🤞

@yhori991
Copy link
Author

We can see all the targets included in the CI/CD now.
The run is not kicked off since I'm not in the whitelist, but this looks promising.

@yhori991
Copy link
Author

My bad. It's already run and green. Thanks you for updating @mattip

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants