Data Engineer with databricks and pysparks
Send a job offer directly to this candidate
Senior Data Engineering Lead with 15+ years of experience designing and modernizing enterprise data platforms across financial services and analytics domains. 5+ years of hands-on experience in Azure Data Engineering stack including Databricks, PySpark, SQL, and Synapse Analytics, with 2+ years of Azure Data Lake (ADLS Gen2) and Azure Data Factory (ADF). Proven expertise in building scalable batch and transformation pipelines using Delta Lake architecture (Bronze–Silver–Gold), optimizing Spark workloads, and leading cross-functional data engineering teams (5 members). Strong background in SQL performance tuning, cloud migration initiatives, and enterprise BI transformation programs.
Senior Data Engineer Lead at Luxoft (2023-04 – Present)
Led data engineering initiatives designing and implementing scalable data pipelines using Databricks and Azure Data Lake with Bronze-Silver-Gold architecture. Built incremental ingestion pipelines, orchestrated ETL workflows, and optimized Spark workloads for high-volume financial datasets.
Data Engineering Lead / BI Modernization at Synechron (2022-09 – 2023-01)
Supported UBS data modernization initiatives using Azure Synapse and Power BI. Developed PySpark and SQL transformations for financial data reconciliation and reporting, integrating SAP ECC and SQL Server datasets into Azure Data Lake.
Senior Database Lead / BI Specialist at Atos-Syntel (2016-05 – 2022-09)
Led SQL and ETL engineering for BARCLAYS global analytics platforms. Designed enterprise data warehouse solutions, developed Power BI dashboards for executive reporting, and implemented performance tuning strategies achieving significant improvements in query execution time.
Database Developer at IBM (2011-03 – 2016-05)
Developed high-performance SQL procedures and reporting solutions. Designed complex data transformation and cleansing scripts, and supported integration of multiple enterprise data sources into centralized analytical systems.
Database Developer at Rave Technologies (2006-10 – 2011-03)
Designed transactional and analytical database systems. Implemented indexing strategies and optimized reporting queries, and integrated SAP and SQL-based data pipelines for enterprise reporting.
Master of Computer Applications in Computer Applications – Jabalpur Engineering College
Bachelor of Science in Mathematics & Computer Science – Khalsa College