Skip to content

Simplify running spark jobs in docker container#7

Open
rohitbansal-tw wants to merge 4 commits intodata-derp:mainfrom
rohitbansal-tw:main
Open

Simplify running spark jobs in docker container#7
rohitbansal-tw wants to merge 4 commits intodata-derp:mainfrom
rohitbansal-tw:main

Conversation

@rohitbansal-tw
Copy link
Copy Markdown
Contributor

@rohitbansal-tw rohitbansal-tw commented Apr 26, 2024

  • Setup docker containers to run spark programs easily, without being concerned of version compatibility/conflicts
  • For support of running spark exercises, created spark.py so that common init block across programs could reside there
  • Updated README to support running on either via docker, or locally on JDK 11 + Spark 3.4.x

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant