-
Notifications
You must be signed in to change notification settings - Fork 38
Description
I am using spark 2.0.1 will these steps still work?
the following command fails, I can see out put of Stage 7 then it disappears and I have to kill the program:
/opt/mapr/spark/spark-2.0.1/bin/spark-submit ./rt_profile_dash.py
[mapr@ae0 spark_music_demo]$ /opt/mapr/spark/spark-2.0.1/bin/spark-submit ./rt_profile_dash.py
^CTraceback (most recent call last):
File "/mapr/my.cluster.com/user/mapr/spark_music_demo/./rt_profile_dash.py", line 95, in
clicked = 1 if sortedclicks.lookup(k)[0] > 0 else 0
File "/opt/mapr/spark/spark-2.0.1/python/lib/pyspark.zip/pyspark/rdd.py", line 2209, in lookup
File "/opt/mapr/spark/spark-2.0.1/python/lib/pyspark.zip/pyspark/context.py", line 933, in runJob
File "/opt/mapr/spark/spark-2.0.1/python/lib/py4j-0.10.3-src.zip/py4j/java_gateway.py", line 1131, in call
File "/opt/mapr/spark/spark-2.0.1/python/lib/py4j-0.10.3-src.zip/py4j/java_gateway.py", line 883, in send_command
File "/opt/mapr/spark/spark-2.0.1/python/lib/py4j-0.10.3-src.zip/py4j/java_gateway.py", line 1028, in send_command
File "/usr/lib64/python2.7/socket.py", line 447, in readline
data = self._sock.recv(self._rbufsize)
File "/opt/mapr/spark/spark-2.0.1/python/lib/pyspark.zip/pyspark/context.py", line 223, in signal_handler
KeyboardInterrupt