Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions tests/spark/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
This application was used to produce the `delta-table` for the delta kuttl test.

It cannot be placed in the delta test folder because beku tries to recurse into it and fails.

To run it, you need a S3 (minio) available. Update the variables below as needed:

S3_ENDPOINT=https://localhost:9000 \
S3_ACCESS_KEY=minioAccessKey \
S3_SECRET_KEY=minioSecretKey \
S3_SKIP_TLS_VALIDATION=true \
DELTA_TABLE_PATH=s3a://trino/delta-table \
./gradlew run
39 changes: 39 additions & 0 deletions tests/spark/build.gradle
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
plugins {
id 'java'
id 'application'
}

group = 'org.example'
version = '1.0-SNAPSHOT'

application {
mainClass = 'org.example.Main'
applicationDefaultJvmArgs = [
'--add-opens=java.base/sun.nio.ch=ALL-UNNAMED'
]
}

repositories {
mavenCentral()
}


dependencies {

// Source: https://mvnrepository.com/artifact/org.apache.spark/spark-sql
implementation 'org.apache.spark:spark-sql_2.12:3.5.5'
// Source: https://mvnrepository.com/artifact/io.delta/delta-spark
implementation 'io.delta:delta-spark_2.12:3.3.2'
// Source: https://mvnrepository.com/artifact/org.apache.hadoop/hadoop-aws
implementation 'org.apache.hadoop:hadoop-aws:3.3.4'
// Source: https://mvnrepository.com/artifact/com.amazonaws/aws-java-sdk-bundle
implementation 'com.amazonaws:aws-java-sdk-bundle:1.12.262'

testImplementation platform('org.junit:junit-bom:5.10.0')
testImplementation 'org.junit.jupiter:junit-jupiter'
testRuntimeOnly 'org.junit.platform:junit-platform-launcher'
}

test {
useJUnitPlatform()
}
Binary file added tests/spark/gradle/wrapper/gradle-wrapper.jar
Binary file not shown.
7 changes: 7 additions & 0 deletions tests/spark/gradle/wrapper/gradle-wrapper.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-9.4.1-bin.zip
networkTimeout=10000
validateDistributionUrl=true
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
248 changes: 248 additions & 0 deletions tests/spark/gradlew

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading