A Study on Traffic Crash Analysis and Geospatial Visualizations Using Virtual Machines and Apache Spark

Chicago is considered one of the congested cities in the United States, with a population of around 3 million (2024). Every year, it is estimated that 100,000 vehicles are involved in accidents. Thus, it is necessary to analyze the crash data to understand the crash trends and patterns, aiming to provide suggestions and recommendations to the Department of Public Safety authorities.  Traffic crashes are a major urban safety issue in places like Chicago, causing injuries, fatalities, and property loss each year. Because crashes occur under many different circumstances, I need to identify which factors increase the risk of fatal outcomes. The city collects extensive crash records, but the data are complex and hard to analyze without appropriate tools. In this project, I performed some major geospatial visualizations on Geospatial Analysis of Fatal Crash Characteristics obtained from data analysis using PySpark in Chicago Traffic Crashes (crashes, people, vehicles, and zero vision fatalities) datasets.

In this project, I used crash-relevant datasets (~ 2GB) such as crashes, people, vehicles, and Vision Zero traffic fatalities owned by the Chicago Police Department (Data Owner).  In my application, I have used the “apache-spark” image (link) for managing one master and two workers. This is done to simplify dependency management and ensure a consistent runtime environment across all nodes. 1 Spark Master and 2 Spark Workers are the core components in my Docker containers.

-------------------------------------------------------------------------------------------------------------------------------------------------------------------

Data:

1. Traffic crashes - crashes https://data.cityofchicago.org/Transportation/Traffic-Crashes-Crashes/85ca-t3if/a bout_data 2
2. Traffic crashes - vehicles https://data.cityofchicago.org/Transportation/Traffic-Crashes-Vehicles/68nd-jvt3/ about_data 
3. Traffic crashes - people https://data.cityofchicago.org/Transportation/Traffic-Crashes-People/u6pd-qa9d/a bout_data
4. Traffic crashes - vision zero Chicago traffic fatalities https://data.cityofchicago.org/Transportation/Traffic-Crashes-Vision-Zero-Chicag o-Traffic-Fatali/gzaz-isa6/about_data

-------------------------------------------------------------------------------------------------------------------------------------------------------------------

Geo-spatial Visualizations:

1. Total number of crashes happening in each city (https://sauravupadhyaya.github.io/traffic_crashes_visualization/crash_markers.html)
2. Fatal Crashes: Cyclists (https://sauravupadhyaya.github.io/traffic_crashes_visualization/people_fatal_ped_cyc_map.html)
3. Fatal Crashes: Pedestrians (https://sauravupadhyaya.github.io/traffic_crashes_visualization/people_fatal_ped_cyc_map.html)
4. Fatal Traffic Accidents Analysis (https://sauravupadhyaya.github.io/traffic_crashes_visualization/fatal_policy_map.html)

Other geo-spatial visualizations:

1. Crashes (top 5 causes): (https://sauravupadhyaya.github.io/traffic_crashes_visualization/crash_top5_causes_map.html)
2.  Visualizing Crashes, Injury, and Fatal: (https://sauravupadhyaya.github.io/traffic_crashes_visualization/crash_admin_map.html)
3. Crashes (Heatmap): (https://sauravupadhyaya.github.io/traffic_crashes_visualization/crash_heatmap.html) 
4. Risk analysis: (https://sauravupadhyaya.github.io/traffic_crashes_visualization/risk_map.html)
5. Vehicles' fatal heavy hit run visualization: (https://sauravupadhyaya.github.io/traffic_crashes_visualization/vehicles_fatal_heavy_hitrun_map.html)