Conversation
Signed-off-by: Enrico Minack <github@enrico.minack.dev>
.github/workflows/ci.yaml
Outdated
| - spark_version: "3.5.5" | ||
| scala_version: "2.13.8" | ||
| - spark_version: "4.0.1-SNAPSHOT" | ||
| scala_version: "2.13.8" |
There was a problem hiding this comment.
Right now, we don't have good integration tests that confirm that all these different version work, (even if they happen to build). Those tests will hopefully get added as a part of: G-Research/spark#45
I have run the demo on 3.3.4, 3.5.3 and 4.1.0 snapshot with the default scala versions, so I believe those to work.
I'm hesitant to add all these other versions until they are tested, because it will be harder for people to know what works. What do you think?
There was a problem hiding this comment.
For the time being, we can setup the demo to run as integration tests in the CI.
There was a problem hiding this comment.
I have moved the new Spark versions into #11.
.github/workflows/ci.yaml
Outdated
| run: mvn --batch-mode dependency:go-offline | ||
| - name: Build | ||
| run: mvn install | ||
| run: mvn --batch-mode install No newline at end of file |
There was a problem hiding this comment.
How hard would it be to add a call to?:
scripts/createImage.sh -s "sparkhome"
that script fails in master right now with spark 3.5, because of some pom changes in our master.
that failure would have been caught earlier if we could have tested it here.
There was a problem hiding this comment.
I have added that.
There was a problem hiding this comment.
Scala 2.13 now successfully builds the image
EnricoMi
force-pushed
the
ci-improve
branch
7 times, most recently
from
826eeae to
ae8d24e
Compare
Signed-off-by: Enrico Minack <github@enrico.minack.dev>
GeorgeJahad
left a comment
GeorgeJahad
left a comment
There was a problem hiding this comment.
This looks good to me, but out of curiosity, why not build the "-snapshot" versions?
|
Could be done if the right branch would be checked out. Room for later improvement. |
2 participants
Uh oh!
There was an error while loading. Please reload this page.