[Snyk] Security upgrade org.apache.spark:spark-core_2.12 from 2.4.4 to 3.2.2

[snyk-io]

Snyk has created this PR to fix 1 vulnerabilities in the maven dependencies of this project.

Snyk changed the following file(s):

• pom.xml

Vulnerabilities that will be fixed with an upgrade:

	Issue	Score	Upgrade
	Command Injection SNYK-JAVA-ORGAPACHESPARK-2774680	900	org.apache.spark:spark-core_2.12: 2.4.4 -> 3.2.2 Major version upgrade No Path Found Mature

---

Important

• Check the changes in this PR to ensure they won't cause issues with your project.
• Max score is 1000. Note that the real score may have changed since the PR was raised.
• This PR was automatically created by Snyk using the credentials of a real user.

---

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.

For more information:
🧐 View latest project report
📜 Customise PR templates
🛠 Adjust project settings
📚 Read about Snyk's upgrade logic

---

Learn how to fix vulnerabilities with free interactive lessons:

🦉 Command Injection

[snyk-io]

This major version upgrade from Spark 2.4.4 to 3.2.2 introduces significant breaking changes. Scala 2.11 support is removed, requiring a move to Scala 2.12. Several APIs in Spark Core and Spark SQL have been changed or removed, and there are notable behavior changes in SQL, Dataset/DataFrame APIs, and UDFs. The upgrade also enables Adaptive Query Execution (AQE) by default, which can alter query plans and performance.

Highlights:

• Scala Version: Support for Scala 2.11 is removed; projects must be compiled with Scala 2.12.
• API Removals: The UserDefinedAggregateFunction (UDAF) API is deprecated in favor of the Aggregator API. The deprecated TaskContext.isRunningLocally method has been removed.

Source: Apache Spark documentation
Recommendation: A thorough migration effort is required. Update the Scala version, replace deprecated UDAF implementations, and validate SQL queries and DataFrame operations for compatibility with Spark 3.x behavior.

Notice 🤖: This content was augmented using artificial intelligence. AI-generated content may contain errors and should be reviewed for accuracy before use.

[snyk-io]

⛔ Snyk checks have failed. 1 issues have been found so far.

Status	Scanner	Critical	High	Medium	Low	Total (1)
⛔	Open Source Security	0	1	0	0	1 issues
✅	Licenses	0	0	0	0	0 issues

💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse.