Fire Accidents Data Analysis with Spark

Used Apache Spark to analyze fire-related accidents in San Francisco in the past two decades. View online notebook here.

Data

Fire Department Calls for Service from 2000 to 2018

Description

• Analyzed spatial/temporal distribution of fire-related accidents using over 4 million fire department data in the last two decades.
• Performed online analytical processing (OLAP) with Spark RDD, Spark Dataframe and Spark SQL.
• Performed spatial clustering of different accidents with PySpark MLlib.
• Analyzed the distribution and variation of different types of fire accidents and identified vulnerable neighborhoods.