Data Engineer | AI Engineer
Send a job offer directly to this candidate
Azure Data Engineer with 4.5 years of experience designing and developing scalable ETL/ELT pipelines using Azure Data Factory, Azure Databricks, ADLS Gen2, PySpark, Delta Lake, and Snowflake. Experienced in data migration, data warehousing, batch processing, performance optimization, and cloud-based analytics solutions supporting enterprise-scale healthcare and financial data platforms.
•Reduced ETL execution time by 40% through Spark optimization and partitioning.
•Processed over 500M+ records monthly utilizing Azure Databricks and ADFManaged data pipelines that supported 50+ downstream reporting datasets.
•Migrated enterprise data from EDP 2.0 to UDAP 3.0, shifting from legacy systems to Azure cloudBuilt and enhanced Azure Data Factory pipelines using Copy Activity, parameterized datasets, and triggers.
•Ingested multi-source data from Oracle databases and ADLS locations into ADLS Gen2.
•Implemented incremental load pipelines using monthly date-based filtering after initial full loads.
•Developed PySpark and Spark SQL notebooks for data cleansing, joins, and schema validation.
•Applied medallion architecture to process data through bronze, silver, and gold layers.
Bachelor of Technology in Mechanical Engineering, 2020
Class 12 - Intermediate Board of Examination, 2016
Class X - Secondary School Examination, 2014
Microsoft Certified: Azure Data Fundamentals (DP-900)
Amazon Certified: AWS Cloud Practitioner
Amazon Certified: AWS Developer Associate
Udemy: ETL and Data Engineering Courses
