Suvro Dutta

Senior Azure Data Engineer with 15+ years of experience architecting, developing, and optimizing large-scale data solutions across financial services, consulting, and technology sectors; delivered platforms supporting 100+ concurrent users at 99.95%+ availability. Certified expert holding Microsoft Azure Solutions Architect Expert (AZ-305), Azure Administrator Associate (AZ-104), Azure DevOps Engineer Expert (AZ-400), Microsoft Fabric Data Engineer Associate, and TOGAF 9 Certified Enterprise Architect. Architected scalable ELT and data warehousing solutions on Microsoft Azure using Azure Data Factory, Azure Data Lake Storage, Azure Blob Storage, and Azure Synapse Analytics to enable governed enterprise analytics.

Designed

Snowflake cloud data platforms by engineering secure schemas, tables, and views and by optimizing Snowflake SQL patterns for high-concurrency analytical workloads. Implemented automated data ingestion and transformation pipelines using Python and SQL with CI/CD practices in Git and Azure DevOps to standardize release governance across environments. Governed enterprise data quality, security, and compliance using Azure Purview, Azure Key Vault, Azure Active Directory, and RBAC to enforce consistent controls across data pipelines and consumption layers.

Proven record designing modern Azure data platforms using Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and Azure Data Lake Storage Gen2 — migrating 20+ legacy workloads with up to 35% processing efficiency gains. Expert in Data Lakehouse architecture using Databricks and Delta Lake, enabling ACID-compliant batch and real-time data processing at enterprise scale, reducing pipeline development time by 40%. Hands-on expertise building ELT pipelines with Python, PySpark, and Spark SQL to process large enterprise datasets, improving data processing throughput by 35% across high-volume workloads.

Architected real-time streaming solutions using Apache Kafka, Azure Event Hubs, and Stream Analytics, processing 500,000+ events/day at sub-second latency for operational analytics. Extensive enterprise database experience across SQL Server, Oracle, PostgreSQL, Azure SQL Database, Cosmos DB, and Snowflake; delivered query performance improvements of up to 50% through tuning and schema optimization. Led cloud migration projects moving legacy ETL workflows and data warehouses to Azure, reducing infrastructure costs by 30% and cutting infrastructure maintenance overhead significantly.

Implemented enterprise data governance, security, and RBAC access controls using Azure Key Vault, Azure Purview, Databricks Unity Catalog, and Azure Active Directory across 200+ datasets. Experienced in DevOps and IaC automation for data platforms using Azure DevOps, Git, ARM Templates, Docker, and AKS — reduced infrastructure provisioning time by 60% across multi-environment deployments.

Senior Azure Data Engineer - Publicis Sapient

(2023-03)

• Designed and implemented enterprise data pipelines using Azure Data Factory and Databricks, integrating 15+ enterprise systems for analytics and reporting and reducing pipeline development time by 40%.
• Designed Snowflake dimensional data models by creating curated tables and secure views and by applying Snowflake SQL optimization techniques aligned to enterprise performance standards.
• Integrated multi-source ingestion into Azure Data Lake Storage and Snowflake using Azure Data Factory, Databricks, and Python to standardize ELT patterns across structured and semi-structured data.
• Automated end-to-end deployment of Azure Data Factory pipelines and Snowflake objects using Git, Azure DevOps CI/CD, and infrastructure-as-code patterns to improve release consistency and governance.
• Optimized pipeline observability and operations by implementing Azure Monitor and Log Analytics alerting for Azure Data Factory and Snowflake workloads to accelerate troubleshooting and incident resolution.
• Governed data security and access controls across Azure and Snowflake by enforcing Azure Key Vault secret management, Azure Active Directory integration, and RBAC policies for least-privilege access.
• Built scalable ingestion frameworks loading structured and semi-structured data into Azure Data Lake Storage Gen2, supporting downstream analytics and ML workloads for 8+ business units.
• Led migration of legacy Teradata and Oracle data warehouse workloads to Azure Synapse, Databricks, and Snowflake, modernizing the analytics platform and reducing infrastructure costs by 30%.
• Developed ELT pipelines using Databricks (PySpark/Scala) and Snowflake SQL, improving data processing performance by 35% and reducing operational complexity.
• Implemented real-time streaming pipelines using Apache Kafka, Azure Event Hubs, and Databricks Structured Streaming, processing 500,000+ events/day at sub-second latency.
• Designed and deployed Delta Lake-based Lakehouse architecture supporting batch and streaming workloads with ACID-compliant storage, improving data reliability by 45%.
• Built automated CI/CD pipelines using Azure DevOps and GitLab with ARM templates and scripting, enabling consistent deployment of ADF pipelines and Databricks notebooks and cutting release cycle time by 50%.
• Developed testing frameworks for data pipelines using PyTest and Great Expectations, improving data quality monitoring and reducing pipeline failures by 45%.
• Implemented enterprise data governance and security policies using Azure Purview, Databricks Unity Catalog, RBAC, and Azure Key Vault across 200+ enterprise datasets.
• Migrated large on-premise PostgreSQL databases to Azure Database for PostgreSQL Flexible Server, reducing infrastructure maintenance overhead by 35%.
• Designed event-driven workflows using Azure Functions and Logic Apps to automate validation, ingestion, and processing — reducing manual intervention by 60%.
• Tuned Spark workloads and optimized Databricks cluster configurations, improving processing throughput by 35% and reducing monthly cloud compute costs.
• Implemented monitoring and alerting using Azure Monitor and Log Analytics, reducing mean time to detect (MTTD) critical pipeline failures by 60%.

Azure Data Engineer - Gentis Solutions

(2017-02 - 2023-02)

• Designed and implemented Azure-based data pipelines using Azure Data Factory, integrating on-premises SQL Server and Oracle systems and supporting 100+ concurrent BI users at 99.95% uptime.
• Developed distributed data processing workflows in Azure Databricks using Spark and PySpark for large-scale transformation, improving throughput by 35%.
• Architected Azure Data Factory orchestration patterns with parameterized pipelines, triggers, and dependency handling to support enterprise-grade scheduling and recoverability.
• Designed Azure Synapse Analytics and Snowflake integration patterns using Azure Data Lake Storage as the staging layer to enable governed ELT and consistent data lineage.
• Implemented data quality controls for ELT pipelines using Python-based validation and Azure Data Factory control-flow patterns to enforce completeness and schema conformance.
• Optimized complex SQL transformations for analytical workloads by tuning queries and modeling curated layers in Azure Synapse Analytics and Snowflake for performance and maintainability.
• Built enterprise data models and reporting datasets within Azure Synapse Analytics, reducing average report refresh latency from 4 hours to under 15 minutes.
• Migrated 10+ legacy SSIS-based ETL pipelines to Azure Data Factory, improving scalability and reducing infrastructure management overhead by 30%.
• Automated data synchronization and workflow orchestration using Azure Logic Apps and Data Factory triggers, reducing manual operational overhead by 40%.
• Integrated Power BI with Azure data sources, enabling self-service analytics for 50+ business stakeholders across 5 business units.
• Implemented enterprise data security and governance including encryption, RBAC policies, data masking, and security policies; achieved GDPR compliance across all enterprise data assets.
• Developed operational automation scripts in Python and PowerShell, reducing monitoring and maintenance task time by 30%.
• Implemented streaming data ingestion pipelines using Azure Event Hubs and Stream Analytics, enabling near-real-time reporting with end-to-end latency under 5 minutes.
• Participated in CI/CD implementation for data pipelines