RayDP crashes with no exception after adding lance-spark-3.5_2.12.jar

[czxxing]

Issue Description

I encountered a runtime error when using RayDP to read Lance files, and no exception logs are available at all. My environment and reproduction steps are as follows:

Environment

• RayDP version: 1.6.5
• Spark version: 3.5.6
• Lance-Spark connector: lance-spark-3.5_2.12.jar
• Deployment mode: Local / Cluster (please select)

Steps to Reproduce

1.Place lance-spark-3.5_2.12.jar into the $SPARK_HOME/jars directory.
2.Initialize a Spark session with RayDP:

import ray
import raydp

ray.init()
spark = raydp.init_spark(
    app_name="LanceReadTest",
    num_executors=2,
    executor_cores=2,
    executor_memory="4GB"
)

df = spark.read.format("lance").load("path/to/lance/file")
df.show()

Questions

1.Does RayDP support reading and writing Lance files?
2.Is there any special configuration required to use the Lance Spark connector in RayDP?
3.How can I fix this runtime error and enable proper logging to debug the issue?