-
Notifications
You must be signed in to change notification settings - Fork 363
Open
Description
I want to up a worker and its master will be mesos-master which I have set up.
however ,when I run docker-compose up ,the console shows this
worker_1 | Exception in thread "main" org.apache.spark.SparkException: Invalid master URL: spark://mesos://zk://zk1:2181 ```
this is docker-compose.yml
```worker:
image: gettyimages/spark:2.2.0-hadoop-2.7
command: bin/spark-class org.apache.spark.deploy.worker.Worker mesos://zk://zk1:2181,zk2:2181,zk3:2181/mesos
hostname: worker
environment:
SPARK_CONF_DIR: /conf
SPARK_WORKER_CORES: 2
SPARK_WORKER_MEMORY: 1g
SPARK_WORKER_PORT: 8881
SPARK_WORKER_WEBUI_PORT: 8081
SPARK_PUBLIC_DNS: localhost
expose:
- 7012
- 7013
- 7014
- 7015
- 7016
- 8881
ports:
- 8081:8081
volumes:
- ./conf/worker:/conf
- ./data:/tmp/data```
brunowego
Metadata
Metadata
Assignees
Labels
No labels