Drusya Suresh

Data Engineer with experience building and supporting scalable cloud-based data engineering solutions using Azure Data Factory, Azure Databricks, Microsoft Fabric, Azure Synapse Analytics, and Azure Data Lake Storage Gen2. Proficient in developing end-to-end ETL/ELT pipelines, automating data ingestion, transforming large datasets using Python, SQL, and PySpark, and delivering curated data for analytics and business reporting. Experienced in implementing Medallion Architecture, Delta Lake, Parquet, Slowly Changing Dimensions (SCD Type 1 & Type 2), incremental data loads, and data validation techniques to improve data quality and consistency.

Hands-on experience working with Microsoft Fabric Lakehouse, Fabric Data Factory, Fabric Warehouse, Dataflow Gen2, and Databricks to develop modern cloud-based data platforms. Strong knowledge of data modeling, data cleansing, performance optimization, pipeline monitoring, troubleshooting, and production support within Azure cloud environments. Skilled in collaborating with business analysts, developers, and cross-functional teams to gather requirements, implement data solutions, and deliver high-quality datasets for reporting and analytics.

Proficient in Git, Azure DevOps, CI/CD, Jira, Confluence, Power BI, and Agile methodologies for efficient software delivery and version control.

Microsoft Certified: Azure Data Fundamentals (DP-900) and Databricks Data Engineer Associate, committed to delivering reliable, scalable, and business-focused data engineering solutions.

Azure Data Engineer at IT Pros LLC (2022-10 – Present)

• Designed, developed, and maintained scalable ETL/ELT pipelines using Azure Data Factory (ADF) and Microsoft Fabric to integrate data from SQL Server, REST APIs, flat files, and cloud storage into Azure Data Lake Storage Gen2 (ADLS Gen2).
• Developed reusable PySpark notebooks in Azure Databricks to perform data transformation, cleansing, enrichment, schema validation, and business rule implementation for enterprise datasets.
• Built and maintained Medallion Architecture (Bronze, Silver, Gold) to support reliable data ingestion, standardized transformations, and business-ready reporting layers.
• Optimized existing pipelines by implementing parameterization, incremental data loading, partitioning, and efficient transformation logic, reducing overall processing time.
• Developed complex SQL queries, views, and stored procedures to validate source-to-target data movement, reconcile record counts, and troubleshoot data discrepancies.
• Worked with Delta Lake and Parquet formats to improve storage efficiency, support ACID transactions, and enhance query performance across large datasets.
• Supported data modeling activities by preparing curated datasets for Azure Synapse Analytics, Microsoft Fabric Warehouse, and Snowflake to meet reporting and analytical requirements.
• Collaborated with business analysts and stakeholders to understand data requirements, translate business rules into technical solutions, and deliver reliable data assets.
• Performed data profiling, quality validation, duplicate detection, null handling, and reconciliation activities to ensure data accuracy before downstream consumption.
• Investigated pipeline failures, notebook execution issues, and data quality exceptions by reviewing ADF logs, Databricks job runs, and monitoring dashboards, ensuring timely issue resolution.
• Assisted in implementing Slowly Changing Dimensions (SCD Type 1 & Type 2) and incremental loading strategies to support historical tracking and efficient data processing.
• Participated in production deployments, code reviews, and release activities using Git and Azure DevOps while following established version control and changing management processes.
• Created technical documentation including data mapping documents, pipeline design specifications, transformation logic, and operational runbooks to support ongoing maintenance.
• Worked in Agile delivery teams, actively participating in sprint planning, backlog refinement, daily stand-ups, sprint reviews, and retrospectives while collaborating with cross-functional teams.
• Supported Power BI reporting initiatives by preparing optimized analytical datasets, validating business metrics, and assisting users with report-related data issues.
• Continuously monitored pipeline performance and identified opportunities to improve reliability, maintainability, and overall data processing efficiency.

Post Graduate Diploma in Supply Chain Management (Logistics) – Fanshawe College

Post Graduate Diploma in Automotive Service Management – Fanshawe College

Bachelor of Technology – Mahatma Gandhi University