Safe Biking in Paris: Modern Data Engineering Pipeline

This project presents a production-ready data engineering pipeline for analyzing bicycle safety in France, particularly in Île-de-France and Paris, using real-world datasets.

By setting up this data engineering and analysis pipeline, you will get:

• Clean, analysis-ready datasets in BigQuery
• Automated EDA reports with key insights and visualizations
• ML-ready features for predictive modeling

Data Engineering Stack:

• Infrastructure as Code: Terraform for reproducible cloud environments
• Containerized Pipeline: Docker for consistent deployment across environments
• Data Transformation: dbt with version control and built-in testing
• Cloud-Native Architecture: Google Cloud Platform (BigQuery + GCS)

Data Quality Issues:

For those looking to reuse this dataset, the project addresses and corrects several data quality issues, including:

• Inconsistent File Naming: e.g., carcteristiques-2021.csv contains spelling errors

• Join Complexity: Requires choosing a granularity (e.g., one row per accident vs. per user)

• Data Quality Fixes:

  • Mixed data types (e.g., latitude column contains strings)
  • Non-breaking spaces disrupting numeric parsing
  • Cross-year schema changes (e.g., vehicle identifiers changed format from 2020 to 2021)
  • Inconsistent time formats (e.g., 4-character time codes with invalid patterns)
  • …