Senior Azure Data Engineer - Infoline System Pvt Ltd
(2024-07)
Client: GEMS Education. Stack: Azure Data Factory, Azure Databricks, PySpark, Spark SQL, Azure Synapse Analytics, Azure SQL, Logic Apps, Python, ADLS Gen2
- Architected and delivered end-to-end ADF pipelines integrating multiple source systems into Azure Data Lake and Azure SQL, enabling organization-wide reporting and analytics for a global education client.
- Built and maintained Databricks notebooks using PySpark and Spark SQL to transform, cleanse, and aggregate student, finance, and operational data at scale, improving data reliability for downstream consumers.
- Implemented Medallion (Bronze/Silver/Gold) architecture in Azure Synapse Analytics, structuring raw-to-curated data layers that accelerated BI reporting readiness.
- Authored and optimized complex T-SQL stored procedures, views, and functions in Azure SQL, improving query performance and ensuring accuracy across integrated reporting systems.
- Designed Logic Apps workflows to automate data movement and pipeline orchestration, reducing manual intervention and cutting operational overhead for recurring data loads.
- Automated data validation, file processing, and exception handling using Python, strengthening pipeline resilience and minimizing production incidents.
- Proactively monitored and resolved ADF and Databricks pipeline failures, implementing alerting mechanisms and documentation that improved mean-time-to-resolution.
- Partnered with cross-functional stakeholders in Agile sprints, aligning pipeline delivery with business timelines and reporting SLAs.
Azure Data Engineer - Microgate Technologies Pvt Ltd
(2022-08 - 2024-04)
Client: IHH Healthcare. Stack: Azure Data Factory, Azure Databricks, PySpark, Python, ADLS Gen2, Azure SQL, Medallion Architecture, Delta Lake
- Owned and built ADF pipelines ingesting healthcare data from multiple source systems into ADLS Gen2, achieving zero data loss and full data integrity across all loads.
- Developed PySpark and Python notebooks in Azure Databricks to process large volumes of patient and operational data, resolving complex edge cases and data quality issues at scale.
- Implemented Bronze/Silver/Gold Medallion architecture in ADLS Gen2, organizing raw, cleansed, and curated layers to make healthcare data progressively analytics-ready.
- Built reusable Python automation scripts for data processing and file ingestion, reducing manual effort by several hours per week and improving pipeline consistency.
- Designed Delta Lake upsert and incremental-load patterns for ACID-compliant processing with full audit trail and time-travel capability.
- Wrote and tuned SQL stored procedures for data validation, reconciliation, and reporting aligned with healthcare analytics requirements.
- Optimized Databricks job performance through PySpark transformation tuning, join optimization, and cluster configuration management to consistently meet SLAs.
- Contributed to code reviews, pipeline documentation, and knowledge-transfer sessions, strengthening team-wide understanding of existing workflows.
SQL Developer - Microgate Technologies Pvt Ltd
(2021-06 - 2022-07)
Client: Wesfarmers. Stack: SQL Server, T-SQL, Power BI, DAX, Power Query, SSIS, Stored Procedures, CTEs, Window Functions
- Developed complex T-SQL queries, stored procedures, functions, and views supporting business reporting and backend logic across multiple Wesfarmers business units.
- Tuned slow-running T-SQL queries through execution plan analysis, index restructuring, and join refactoring to sustain performance under high production loads.
- Built advanced analytical T-SQL logic using CTEs, window functions, and ranking functions feeding directly into reporting dashboards.
- Developed interactive Power BI dashboards, transforming SQL Server data via Power Query and building DAX measures and KPIs for business stakeholders.
- Designed Power BI data models with proper relationships, hierarchies, and filter contexts, ensuring accurate drill-down and cross-filter behavior.
- Collaborated with analysts and stakeholders through iterative feedback cycles to align SQL logic and Power BI visuals with business reporting requirements.
