[SPARK-52506][K8S] Allow migrating to fallback storage only#95

Open

EnricoMi wants to merge 458 commits intomasterfrom

allow-disabling-peer-decommisioning

This pull request is big! We're only showing the most recent 250 commits

Commits on Jan 29, 2026

Commits on Jan 30, 2026

[SPARK-55282][PYTHON][CONNECT] Avoid using worker_util in the Driver-side

ueshin
authored and
HyukjinKwon
committed
[SPARK-55263][PYTHON][INFRA] Upgrade Python linter from 3.11 to 3.12 in CI

Yicong-Huang
authored and
HyukjinKwon
committed
[SPARK-55281][PYTHON] Add ipykernel and IPython to mypy optional package list

gaogaotiantian
authored and
HyukjinKwon
committed
[SPARK-55266][INFRA] Add pre-commit hooks for format/lint

gaogaotiantian
authored and
HyukjinKwon
committed
[SPARK-55276][BUILD] Upgrade `scala-maven-plugin` to 4.9.9

dongjoon-hyun
authored and
HyukjinKwon
committed
[SPARK-55272][BUILD] Upgrade SBT to 1.12.1
LuciferYang
committed
[SPARK-55273][SQL] Replace `ParquetFileReader.open().getFooter()` with `readFooter()` to avoid unnecessary operations in `ParquetFooterReader`
LuciferYang
committed
[SPARK-55237][SQL] Suppress annoying messages when looking up nonexistent DBs

pan3793
authored and
LuciferYang
committed
[SPARK-55031][SQL] Add vector avg/sum aggregation function expressions

zhidongqu-db
authored and
cloud-fan
committed
[SPARK-55236][CORE] Address unexpected exception in some CoarseGrainedExecutorBackendSuite test cases

ChuckLin2025
authored and
Ngone51
committed
[SPARK-55277][SQL] Add `protobuf_funcs` group for Protobuf SQL functions
yaooqinn
committed
Revert "[SPARK-55277][SQL] Add `protobuf_funcs` group for Protobuf SQL functions"
yaooqinn
committed
[SPARK-55277][SQL] Add `protobuf_funcs` group for Protobuf SQL functions
yaooqinn
committed
[SPARK-49110][SQL] Simplify SubqueryAlias.metadataOutput to always propagate metadata columns
cloud-fan
committed
[SPARK-55133][CONNECT] Fix race condition in IsolatedSessionState lifecycle management
cloud-fan
committed
[SPARK-55256][SQL] Support IGNORE NULLS / RESPECT NULLS for array_agg and collect_list
yaooqinn
committed
[SPARK-55239][CONNECT][YARN] Allow to launch SparkConnectServer in YARN cluster mode
sarutak
committed
[SPARK-55279][SQL] Add `sketch_funcs` group for DataSketches SQL functions
yaooqinn
committed

Commits on Jan 31, 2026

Commits on Feb 1, 2026

[SPARK-55123][SS] Add SequentialUnionOffset for tracking sequential source processing

ericm-db
authored and
anishshri-db
committed
[SPARK-55105][SS] Add Integration Test for Join Operator

zifeif2
authored and
anishshri-db
committed
[SPARK-55287][INFRA] Consolidate steps in `lint`

zhengruifeng
authored and
HyukjinKwon
committed
[SPARK-55176][PYTHON] Extract `arrow_to_pandas` converter into ArrowArrayToPandasConversion

Yicong-Huang
authored and
HyukjinKwon
committed
[SPARK-55284][PYTHON][TEST] Move mypy-data related configs to the script

gaogaotiantian
authored and
HyukjinKwon
committed
[SPARK-55285][SQL][PYTHON] Fix the initialization of `PythonArrowInput`

zhengruifeng
authored and
HyukjinKwon
committed
[SPARK-55283][PYTHON][PS][TESTS] Add a new argument ignore_null to assert_eq

gaogaotiantian
authored and
HyukjinKwon
committed
[SPARK-55286][INFRA] Add test summary to GitHub Actions for better failure visibility

yaooqinn
authored and
HyukjinKwon
committed
[SPARK-55293][PS][TESTS] Avoid using old offset aliases

ueshin
authored and
HyukjinKwon
committed

Commits on Feb 2, 2026

Commits on Feb 3, 2026

Commits on Feb 4, 2026

Commits on Feb 5, 2026

Commits on Feb 6, 2026

Commits on Feb 7, 2026

Commits on Feb 8, 2026

Commits on Feb 9, 2026

Commits on Feb 10, 2026

[SPARK-55385][CORE][SQL][FOLLOWUP] Rename preservesDistribution to preservesPartitionSizes

cloud-fan
authored and
zhengruifeng
committed
[SPARK-55369][SPARK-55332][PYTHON][INFRA] Setup ASV benchmark

fangchenli
authored and
zhengruifeng
committed
[SPARK-55456][BUILD] Upgrade `zstd-jni` to 1.5.7-7
dongjoon-hyun
committed
[SPARK-55459][PYTHON] Fix 3x performance regression in applyInPandas for large groups

Yicong-Huang
authored and
zhengruifeng
committed
[SPARK-55405][PYTHON][TESTS][FOLLOWUP] Add safe=False tests for PyArrow Array.cast()

Yicong-Huang
authored and
zhengruifeng
committed
[SPARK-55457][INFRA] Add `logging.warning` check in ruff

gaogaotiantian
authored and
zhengruifeng
committed
[SPARK-55460][PYTHON] Remove E203 from ruff's ignore list

gaogaotiantian
authored and
zhengruifeng
committed
[SPARK-55458][PYTHON][TESTS] Apply the new test pattern for newly added tests

gaogaotiantian
authored and
zhengruifeng
committed
[SPARK-55411][SQL] SPJ may throw ArrayIndexOutOfBoundsException when join keys are less than cluster keys

pan3793
authored and
peter-toth
committed
[SPARK-55451][SQL] Cursors must start collecting results on OPEN, not first FETCH

srielau
authored and
gengliangwang
committed

Commits on Feb 11, 2026

Commits on Feb 12, 2026

[SPARK-55490][PS] Make `groupby(as_index=False)` include a grouping that is not in the DataFrame with pandas 3

ueshin
authored and
HyukjinKwon
committed
[SPARK-54805][SS][PYTHON][FOLLOW-UP] Add test_tws_tester to modules

gaogaotiantian
authored and
zhengruifeng
committed
[SPARK-55483] Fix NPE in PivotFirst when pivot column is a non-atomic type with null values

miland-db
authored and
cloud-fan
committed
[MINOR][DOCS] Fix several nit issues in documentation

anshulbaliga7
authored and
pan3793
committed
[SPARK-55462][PYTHON] Support UserDefinedType in convert_numpy

fangchenli
authored and
zhengruifeng
committed
[SPARK-55487][PYTHON][TESTS] Detect dangling tests

gaogaotiantian
authored and
zhengruifeng
committed
[SPARK-55498][BUILD][TESTS] Upgrade `oracle-free` docker image to `23.26.1-slim`

sarutak
authored and
dongjoon-hyun
committed
[SPARK-55495][CORE] Fix `EventLogFileWriters.closeWriter` to handle `checkError`
dongjoon-hyun
committed
[SPARK-54784][ML][DOCS] Document the security policy on ml models

zhengruifeng
authored and
dongjoon-hyun
committed
[SPARK-54740][PYTHON] Start faulthandler early in daemon mode

gaogaotiantian
authored and
HyukjinKwon
committed
[SPARK-55020][PYTHON][FOLLOW-UP] Move release into disable gc protection to prevent deadlock

gaogaotiantian
authored and
HyukjinKwon
committed

Commits on Feb 13, 2026

Commits on Feb 14, 2026

[SPARK-55526][WEBUI][TESTS] Add `glob` package to `ui-test`

sarutak
authored and
dongjoon-hyun
committed
[SPARK-55522][WEBUI] Allow inline scripts, event handlers and styles in Spark UI with Content-Security-Policy

sarutak
authored and
dongjoon-hyun
committed
[SPARK-55479][SQL] Fix style issues in SparkShreddingUtils

manuzhang
authored and
dongjoon-hyun
committed

Commits on Feb 15, 2026

Commits on Feb 16, 2026

[SPARK-55372][SQL] Fix `SHOW CREATE TABLE` for tables / views with default collation

ilicmarkodb
authored and
cloud-fan
committed
[SPARK-55500][SQL] Fix analyzer cycle between `ApplyDefaultCollation`, `ExtractWindowExpressions` and `CollationTypeCasts`

ilicmarkodb
authored and
cloud-fan
committed
[SPARK-55261][GEO][SQL] Implement Parquet read support for Geo types

uros-db
authored and
cloud-fan
committed
[SPARK-55541][GEO][SQL] Support Geometry and Geography in catalyst type converters

uros-db
authored and
cloud-fan
committed
[SPARK-55521][CONNECT] Make exception constructors for Spark Connect client private[spark]

heyihong
authored and
cloud-fan
committed
[SPARK-55496][K8S] Support re-use of scaled PVCs

Dzeri96
authored and
dongjoon-hyun
committed

Commits on Feb 17, 2026

Commits on Feb 18, 2026

Commits on Feb 24, 2026

Disable peer decommissioning when shuffle max disk size is 0
EnricoMi
committed