Deepti Gupta

Senior Azure Data Engineer and Data Platform Architect with 12+ years of experience designing scalable cloud data platforms, enterprise data pipelines, and big data analytics solutions. Deep expertise in Azure Data Factory, Azure Databricks, Azure Data Lake, PySpark, and distributed data processing. Proven track record of leading large-scale cloud migrations, building high‑performance ETL/ELT pipelines, implementing NoSQL and big data architectures, and enabling advanced analytics for global organizations.

Experienced in collaborating with cross‑functional teams, defining architecture standards, and implementing data governance, security, and compliance strategies for enterprise data ecosystems.

Senior Azure Data Engineer at MetLife Insurance (2024-04 – Present)

• Designed scalable ETL/ELT pipelines using Azure Synapse, Azure Data Factory and Databricks.
• Built Customer 360 and Policy Data Integration platform for insurance data.
• Implemented hash-based CDC, Delta MERGE and SCD Type 2.
• Developed PySpark transformations, data cleansing and reconciliation frameworks.
• Designed and optimized SQL Server stored procedures, views and functions.
• Performed SQL query tuning, indexing and execution plan optimization.
• Supported Azure DevOps CI/CD deployments for Synapse, Databricks and SQL database objects.
• Implemented monitoring, logging, production deployment support and performance tuning.

Data Engineer / Technical Architect at ASDA Ecommerce (2021-12 – 2024-03)

• Led migration of enterprise ecommerce datasets from legacy platforms to Azure cloud.
• Designed scalable ETL pipelines using Azure Data Factory and Azure Databricks.
• Built centralized data lake architecture supporting analytics and reporting workloads.
• Implemented data validation frameworks ensuring zero data loss during migration.
• Defined architecture standards including pipeline design patterns, monitoring, logging, and error‑handling frameworks.
• Collaborated with business analysts, QA teams, infrastructure teams, and leadership stakeholders to translate business goals into scalable technical architecture.
• Implemented RBAC, encryption, and data governance practices to safeguard sensitive data.
• Improved pipeline efficiency and reduced processing times through optimized Spark transformations.

Big Data Engineer at Connected Chillers IoT Platform – Global Confectionery Client (2016-01 – 2021-11)

• Developed IoT analytics platform monitoring chiller performance to prevent chocolate spoilage.
• Processed high‑volume sensor datasets using Hadoop, Cloudera, and Python analytics pipelines.
• Implemented MongoDB NoSQL database for storing semi‑structured sensor telemetry data.
• Designed indexing strategies (deviceId, timestamp) and aggregation queries to optimize performance.
• Built predictive analytics workflows to detect temperature anomalies and spoilage risk patterns.
• Implemented alerting and monitoring mechanisms notifying stakeholders of abnormal conditions.
• Developed microservices-based architecture integrating sensors, APIs, and analytics systems.
• Collaborated with IoT engineers, data scientists, and product teams to deliver a scalable analytics solution.

Software Engineer at Walmart Retail Technology Solutions (2012-01 – 2015-12)

• Built enterprise web applications supporting retail operations monitoring.
• Developed automation scripts improving detection of financial transaction mismatches.
• Built retail visualization features enabling store managers to optimize product placement.