Shubham Parate

Data Engineer with 4 years of experience in building and optimizing data pipelines, ETL workflows, and cloud-native solutions on Google Cloud Platform (GCP). Skilled in BigQuery, Dataflow, Pub/Sub, Cloud Storage, and SQL with a strong foundation in Java, Python, and backend systems. Adept at enabling real-time data processing, scalable data models, and ensuring data quality for analytics and ML use cases.

Lead Data Engineer - Persistent System Limited - Pune, Maharashtra, India

(2025-10)

• Designed and deployed data pipelines using GCP Dataflow & Pub/Sub to process real-time transactional data (50K events/minute).
• Migrated on-prem SQL warehouse to BigQuery, reducing query runtime by 40%.
• Built automated ETL workflows using Cloud Composer (Airflow), orchestrating 10+ pipelines daily.

Senior Data Engineer - Persistent System Limited - Pune, Maharashtra, India

(2024-07 - 2025-10)

• Optimized BigQuery datasets & partitioning, cutting down storage costs by 25%.
• Collaborated with data analysts and ML engineers to provide cleaned datasets for predictive modeling.
• Implemented data ingestion layer to push logs into GCP Cloud Storage for further processing.

Data Engineer - Persistent System Limited - Pune, Maharashtra, India

(2022-07 - 2024-07)

• Developed REST APIs in Java/Spring Boot integrated with PostgreSQL for analytics systems.
• Automated data validation scripts in Python, ensuring high data quality across multiple sources.
• Worked with Docker & Git CI/CD pipelines to deploy services in cloud environments.

B.Tech - Computer Science and Engineering - KIT's College of Engineering (Autonomous) Kolhapur (2018 - 2022)