Subhashini

Azure Data Engineer with 4+ years of experience designing and delivering enterprise-scale data platforms using Azure Data Factory, Databricks, Synapse Analytics, and Microsoft Fabric. Proven expertise in building Delta Lakehouse architectures, metadata-driven ETL frameworks, and scalable PySpark pipelines across financial services and retail clients including Innovative Logical Solutions and Commerzbank. Strong background in data governance, implementing Unity Catalog, RBAC, and GDPR-compliant solutions.

DP-203 certified, with a track record of optimizing data pipelines, enabling real-time analytics, and supporting business-critical decision-making.

Data Engineer - Innovative Logical Solutions

(2024-08)

• Architected metadata-driven ADF pipelines on Microsoft Fabric integrating structured and semi-structured data sources, enabling end-to-end pipeline traceability across the data platform.
• Designed Databricks and Unity Catalog data processing framework with full data lineage tracking across bronze, silver, and gold layers, supporting governance and GDPR compliance.
• Led requirements workshops with business stakeholders across trading and supply chain teams, translating KPIs into Synapse Analytics models and delivering same-day data insights to business users.
• Used PySpark for distributed data transformations and optimized ETL processes through caching, parallel processing, and partition pruning, reducing pipeline runtime while maintaining accuracy.
• Implemented RBAC and Unity Catalog governance controls across 3 environments, achieving full audit-trail coverage for sensitive retail datasets.
• Built metadata-driven data pipelines using Azure Data Factory, Delta Lake, and Azure Purview to support schema evolution and dynamic data source handling.
• Designed and deployed Power BI dashboards integrated with Azure SQL Database and Dataverse, enabling real-time KPI monitoring and data-driven decision-making.
• Developed and deployed Microsoft Fabric data pipelines for unified data ingestion across operational and analytical systems in hybrid and multi-cloud environments.
• Integrated CI/CD pipelines using Azure DevOps to automate build, test, and release workflows for ETL components across development and production environments.
• Strengthened data quality and operational resilience by improving ETL reliability and data availability to support business reporting and advanced analytics across enterprise systems.

Data Engineer - VA Solutions

(2023-08 - 2024-05)

• Migrated legacy SSIS and SSAS workflows to ADF and Databricks, reducing pipeline maintenance effort and improving SAP source integration throughput.
• Architected hybrid Lakehouse on ADLS, Synapse, and Cosmos DB for near-real-time risk analytics, processing high-volume daily financial events for trading dashboards.
• Improved ETL throughput and data consistency across multiple ingestion pipelines by implementing PySpark broadcast joins, checkpointing, and adaptive partition pruning.
• Built Python and Airflow monitoring and alerting system for production pipelines, reducing mean time to detect and resolve failures across the data platform.
• Delivered Power BI dashboards integrating SAP BW and Azure SQL, providing trading and operations teams with real-time drilldown analytics.
• Implemented IAM and RBAC policies for sensitive datasets, ensuring full compliance with enterprise security standards across internal audit cycles.

Data Engineer - Commerzbank

(2022-11 - 2023-07)

• Partnered with business stakeholders to gather requirements and design Azure-based data warehouse solutions using Databricks, Unity Catalog, and Synapse Analytics.
• Designed metadata-driven pipelines using Delta Lake and ADF, enhancing code modularity and improving schema adaptability across changing data environments.
• Built modern Data Lakehouse architecture using ADLS and Delta Lake, supporting both batch and real-time data workloads under a unified governance framework.
• Optimized ETL pipelines using PySpark caching, checkpointing, and partition tuning, improving performance and scalability across complex enterprise workloads.
• Implemented IAM and RBAC access control policies to secure highly sensitive datasets while maintaining flexible access for developers and analysts.
• Leveraged Python automation to build data quality frameworks and automate ETL pipeline runs with dynamic exception tracking and real-time notification systems.
• Created interactive Power BI dashboards integrating SAP BW and Azure SQL, providing business management with dynamic drilldowns and trend analytics.
• Engineered secure data solutions using Unity Catalog and Synapse Analytics, ensuring seamless data cataloguing and enhanced data lineage visibility across multi-tenant environments.

BI Developer - State Auto Insurance

(2022-05 - 2022-10)

• Engaged with business stakeholders to gather BI and reporting requirements, translating business KPIs into structured data warehouse models.
• Built optimized ETL pipelines using SSIS and Alteryx, integrating SAP and SQL data sources to deliver accurate, timely, and consistent data feeds for enterprise analytics.
• Developed Power BI and Tableau dashboards featuring dynamic visualizations, drill-down analysis, and automated data refresh processes.
• Created efficient SSAS cubes with advanced MDX calculations to enable fast multidimensional querying and improve aggregation performance.
• Applied DAX and SQL scripting to optimize complex report calculations and reduce data retrieval latency across dashboards.
• Automated end-to-end report scheduling, alert notifications, and audit processes using Power BI Service and SQL Server Agent.
• Maintained strong data governance and security policies by implementing access controls, metadata documentation, and user-level permissions.

Master's - MSc Digital Marketing - Ravensbourne University London (2024 - 2025)