spark-sqs-receiver for pyspark+boto #4
Description
Hi,
Is it possible to create a similar receiver for pyspark (structured streaming) to receive stream from amazon SQS. Stack shows a similar question I asked.
Can we at all have a costume receiver with PySpark? In a nut shell I want to receive AWS-SQS stream in Spark context(streaming) in python and store it in Hadoop.