Vaibhav Ambatkar

Certified Azure Data Engineer with 3.4+ years of experience building scalable data platforms on Microsoft Azure. Skilled in developing end-to-end data pipelines using Azure Data Factory, Azure Databricks, PySpark, and ADLS Gen2 to support analytics and reporting workloads. Experienced in Lakehouse architecture using Delta Lake with Bronze, Silver, and Gold layers for efficient data ingestion, transformation, and curation.

Strong expertise in incremental data processing, data validation, and pipeline performance optimization. Knowledgeable in data governance and access control using Unity Catalog and RBAC. Focused on delivering reliable, analytics-ready datasets that drive business insights and data-driven decision making.

Azure Data Engineer at Cloudsyra Pvt. Ltd (2023-07 – Present)

Architected scalable data ingestion pipelines and built analytics-ready datasets supporting business intelligence solutions.

• Architected scalable data ingestion pipelines using Azure Data Factory to integrate data from REST APIs, databases, and cloud sources into ADLS Gen2.
• Developed PySpark notebooks in Azure Databricks for data cleansing, transformation, deduplication, and schema standardization.
• Implemented Lakehouse architecture using Delta Lake with Bronze, Silver, and Gold layers to structure raw, refined, and curated datasets.
• Designed incremental data processing using watermark columns and Delta MERGE operations to maintain continuously updated datasets.
• Optimized pipeline performance using partitioning strategies, query optimization, and monitoring of Databricks jobs and Azure Data Factory executions.
• Implemented data governance and security using Unity Catalog, Azure RBAC policies, and Azure Key Vault for secure access and credential management.
• Built analytics-ready datasets and data models supporting business intelligence and reporting solutions for data-driven decision making.

Graduate Trainee Engineer at Cloudsyra Pvt. Ltd (2023-01 – 2023-07)

Supported Azure data pipeline development and maintained ETL workflows with focus on data quality and production stability.

• Supported Azure data pipeline development, assisting with data ingestion and transformation in an Azure Data Lake environment.
• Assisted in building and maintaining ETL workflows using Azure Data Factory and Azure Databricks for integrating data from multiple sources.
• Performed SQL-based data validation, profiling, and data quality checks to ensure accuracy, consistency, and completeness of datasets.
• Contributed to CI/CD pipelines, deployment activities, and release management to support reliable production data workflows.
• Monitored pipeline executions, troubleshot failures, and resolved production issues to maintain stable and efficient data processing systems.

Bachelor of Technology in B.Tech – DR.PDKV (2023)