Chaitanya Yadav

Accomplished Azure Data Engineer with 4+ years of hands-on experience in designing, building, and managing scalable data pipelines and cloud data platforms using Microsoft Azure. Specialized in developing ETL/ELT workflows using Azure Data Factory (ADF), SQL, Python, and PySpark to ingest and transform large volumes of structured and semi-structured data. Built modular, reusable data pipelines using ADF and PySpark to handle batch and near real-time processing from cloud and on-premise data sources.

Migrated legacy ETL frameworks (SSIS) to cloud-native solutions on Azure, improving scalability, maintainability, and cost-efficiency. Designed and implemented Bronze-Silver-Gold layer architecture in Azure Data Lake for clean data separation, transformation tracking, and auditability. Led the implementation of data quality frameworks including validation rules, profiling checks, exception handling, and automated alerts using Azure Monitor and Log Analytics.

Proficient in implementing data lake and data warehouse architectures using Azure Data Lake Storage Gen2 and Azure Synapse Analytics, supporting enterprise-level analytics and reporting. Extensive experience in dimensional modeling (star and snowflake schemas), building curated data models to serve self-service BI tools such as Power BI.

Azure Data Engineer at Exner Technologies (2022-01 – Present)

Azure Data Engineer - Project 1: Molina Healthcare at Exner Technologies (2024-03 – Present)

• Designed and developed scalable ETL/ELT pipelines using Azure Data Factory to ingest member enrollment, claims, provider, and pharmacy data from on-premises and cloud sources into Azure Data Lake Storage Gen2.
• Built high-performance data transformation workflows in Azure Databricks using PySpark to cleanse, standardize, and aggregate healthcare datasets for regulatory and operational reporting.
• Developed and optimized dimensional models and fact tables in Azure Synapse Analytics to support claims analysis, care management, and provider performance dashboards.
• Built lakehouse solutions in Microsoft Fabric to integrate curated healthcare data and provide a unified reporting layer for business and analytics teams.
• Configured secure access to ADLS Gen2, Azure Synapse, and Fabric using role-based access control, managed identities, and Key Vault to protect sensitive healthcare data and ensure compliance.
• Scheduled and monitored end-to-end pipelines in Azure Data Factory and Fabric, troubleshooting failures and tuning workloads to meet SLA requirements for daily and monthly data loads.
• Collaborated with business analysts, data architects, and healthcare stakeholders to gather requirements and deliver data solutions for member analytics, utilization management, and regulatory reporting.

Azure Data Engineer - Project 2: Aviva Insurance at Exner Technologies (2022-01 – 2024-02)

• Designed and developed end-to-end ETL pipelines using Azure Data Factory to ingest policy, claims, customer, and underwriting data from multiple on-premises and cloud sources.
• Built scalable data transformation workflows in Azure Databricks using PySpark to cleanse, standardize, and enrich insurance datasets for downstream analytics.
• Developed and optimized complex SQL Server stored procedures, views, and T-SQL scripts for incremental loading, reconciliation, and data validation.
• Created data quality and exception-handling frameworks using Python to identify missing records, duplicate policies, and inconsistent claims information.
• Implemented Delta Lake architecture in Azure Databricks to support ACID transactions, schema evolution, and high-performance processing of large insurance datasets.
• Designed dimensional data models and curated datasets to support reporting on policy renewals, claims settlement, customer retention, and risk analysis.

B.Tech. – JNTU Anantapur