Rajeshwari

Azure Data Engineer with 4.3+ years of experience designing, developing, and optimizing cloud-based data platforms using Microsoft Azure technologies. Proficient in building end-to-end ETL/ELT pipelines using Azure Data Factory (ADF), Azure Databricks (PySpark), Azure Data Lake Storage Gen2 (ADLS), Azure Synapse Analytics, and Delta Lake, with deep expertise in Lakehouse architecture and Medallion Architecture (Bronze–Silver–Gold) for managing large-scale structured and semi-structured datasets. Skilled in Python, PySpark, and SQL for large-scale data transformations, incremental data processing, metadata-driven pipelines, and enterprise data quality validation frameworks. Delivered reliable analytics platforms across Pharma &

• Life Sciences, Travel &

• Hospitality, Energy &

• Utilities, and Media &

• Streaming domains, with strong command of data governance, CI/CD automation via Azure DevOps, and pipeline monitoring using Azure Monitor and Logic Apps.

Azure Data Engineer at HCL Technologies (2023-08 – Present)

Currently working as Azure Data Engineer on pharma and travel analytics projects

• Designed and implemented ADF pipelines to ingest pharmaceutical datasets from APIs, clinical research databases, and flat files
• Built PySpark transformation pipelines in Azure Databricks to process clinical trial and drug production datasets
• Implemented Medallion Architecture (Bronze–Silver–Gold) in ADLS Gen2 for structured data processing
• Developed Delta Lake tables to support incremental loads, schema evolution, and historical data tracking
• Implemented data validation frameworks including schema validation, null checks, and duplicate detection
• Built Azure Functions to automate metadata-driven pipeline triggers and workflow automation
• Designed analytics-ready datasets for drug performance reporting and compliance monitoring
• Implemented data security using RBAC and Azure Key Vault for protecting sensitive healthcare datasets
• Optimized Spark jobs using partitioning, caching, and efficient join strategies
• Monitored pipeline health and alerts using Azure Monitor dashboards
• Developed ADF pipelines to ingest booking transactions, property listings, and customer interaction datasets
• Implemented PySpark transformations in Azure Databricks to process large-scale booking and travel datasets
• Designed ADLS folder structures with efficient partitioning strategies for large-scale data storage
• Built Delta Lake tables for incremental processing of booking transactions and customer data
• Developed SQL scripts for booking reconciliation, pricing validation, and transaction monitoring
• Built Power BI datasets for travel analytics dashboards and revenue reporting
• Automated CI/CD deployment pipelines using Azure DevOps for ADF and Databricks notebooks
• Optimized Spark workloads using repartitioning and caching techniques
• Delivered curated datasets for data science teams to build travel demand forecasting models
• Provided production support for resolving pipeline failures and performance issues

Data Engineer at Wipro Technologies (2021-12 – 2023-08)

Worked on energy and utilities data platform and media & streaming analytics projects

• Developed ADF pipelines to ingest energy production datasets from IoT devices, operational systems, and APIs
• Built PySpark transformation pipelines in Azure Databricks to process large-scale energy consumption datasets
• Implemented Delta Lake architecture with Bronze, Silver, and Gold layers for structured data processing
• Developed data validation frameworks for ensuring accuracy of energy production metrics
• Implemented incremental loading frameworks for handling high-frequency IoT datasets
• Designed data models supporting energy consumption reporting and operational analytics
• Implemented RBAC-based security and encryption mechanisms for protecting energy data
• Collaborated with data scientists and analysts to deliver analytics-ready datasets
• Automated pipeline monitoring and alerting using Logic Apps and Azure Monitor
• Improved Spark performance through optimized joins, caching, and partition strategies
• Developed ADF pipelines to ingest streaming logs and user interaction datasets from multiple sources
• Built PySpark transformation workflows to process large-scale streaming datasets
• Implemented Delta Lake incremental processing for managing user interaction data
• Developed data transformation logic for aggregating user viewing metrics and engagement analytics
• Built fact and dimension models in Azure Synapse Analytics for streaming analytics dashboards
• Implemented data validation frameworks to ensure data accuracy and completeness
• Delivered curated datasets for machine learning models supporting content recommendation systems
• Optimized Spark workloads using partitioning and broadcast joins
• Provided analytics-ready datasets for Power BI dashboards and executive reporting

Bachelor of Science (B.Sc.) in Computer Science – Karnataka University (2021)