Sudip Roy
Data Engineer with experience delivering end-to-end data solutions using Azure Databricks, PySpark, and SQL
Talent
KolkataMember since June 1, 2026
PySparkSQLScalaETL/ELT PipelinesData QualityDatabricksUnity CatalogAzure DatabricksAzure Data Factory
·
Hire this person
Send a job offer directly to this candidate
PySparkSQLScalaETL/ELT PipelinesData Quality
About
Data Engineer with experience delivering end-to-end data solutions using Azure Databricks, PySpark, and SQL. Proven ability to design scalable ETL/ELT pipelines, integrate multi-source data, and enhance data quality, governance, and performance. Certified in Databricks and Azure, with strong analytical and problem-solving skills, and experience working in Agile delivery environments.
Experience
Data Engineer Analyst at Capgemini (2023-11 – Present)
- Designed and developed scalable data pipelines using Azure Databricks (PySpark).
- Built and optimized ETL/ELT workflows using SQL and Databricks notebooks.
- Orchestrated workflows using Azure Data Factory (ADF).
- Implemented data validation and transformation rules for quality and consistency.
- Contributed to Unity Catalog migration for governance and access control.
- Optimized pipelines for performance and scalability.
- Supported end-to-end lifecycle: analysis, development, testing, deployment.
- Collaborated in Agile teams and resolved technical issues.
- Documented workflows and conducted knowledge transfer sessions.
Education
MBA in Analytics & Data Science – Manipal University Jaipur (2024)
B.Tech in Computer Science & Engineering – IEM Kolkata (2023)
Skills
PySparkand SQL
