Jaquet Spark streaming from kafka(json) to s3(parquet) - Excatly once end2end https://medium.com/@eranshemesh/jaquet-saving-your-mass-of-events-b9d1d5f16c5