Sudha Sinha

Data Engineering professional with over 4 years of expertise in Azure Data Engineering, leveraging Azure Databricks, Azure Data Factory, Azure Synapse Analytics, PySpark, Spark SQL, ADLS Gen2, and Delta Lake to build scalable and high-performance data solutions. Driving enterprise data engineering initiatives at Tata Motors as a Big Data Engineer, enabling large-scale ingestion, transformation, migration, and analytics workloads across cloud-based platforms. Achieved a 50% reduction in processing time, accelerated migration activities by 70%, and engineered ingestion frameworks managing 216+ GB of daily data throughput through optimized Spark and Databricks ecosystems.

Specialized in building Lakehouse architectures, Medallion-based data platforms, dimensional data models, real-time and batch ETL frameworks, cloud migration solutions, and enterprise-scale data integration pipelines. Distinguished by combining strong engineering execution, cloud modernization expertise, and performance optimization capabilities to deliver reliable, scalable, and business-focused data ecosystems.

Big Data Engineer at Tata Motors (2024-01 – Present)

• Driving the automation of the data extraction layer & optimizing Spark workloads, resulting in a 50% reduction in daily processing time from 8 hours to 4 hours.
• Designing and implementing enterprise-scale data integration solutions using Azure Data Factory, Azure Databricks, ADLS Gen2, and Synapse Analytics, supporting and enabling the seamless execution of 200+ production pipelines.
• Building and enhancing an ADLS automation framework that is streamlining data migration for reporting, improving pipeline reliability, and reducing overall migration time by 70%.
• Developing and scaling an incremental ETL pipeline using Databricks Auto Loader, ingesting and processing over 216 GB of data daily from Amazon S3 into Delta Lake while ensuring schema evolution, checkpointing, and fault-tolerant data processing.
• Automated JSON data ingestion from AWS S3 to the Bronze layer using Databricks Auto Loader (Cloud Files), enabling scalable incremental data extraction with schema evolution and reducing daily processing time by 50%.
• Developed a real-time data streaming pipeline using Databricks Structured Streaming, ingesting vehicle events from Kafka and enriching them with historical vehicle master data stored in Delta Lake, leveraging checkpointing and stateful processing for reliable stream execution.

PGET-Data Engineer at Tata Motors (2023-01 – 2023-12)

• Engineered scalable end-to-end ETL/ELT pipelines and data migration solutions using Azure Databricks, Azure Data Factory (ADF), and Azure Synapse Analytics, leveraging Medallion Architecture to enhance data reliability and accessibility.
• Orchestrated seamless integration of data from Oracle, SQL Server, REST APIs, Amazon S3, and cloud storage platforms through Azure Data Factory and Databricks, while implementing data quality validations to improve data accuracy.
• Designed and optimized relational and dimensional data models that enabled enterprise reporting, advanced analytics, and improved business decision-making.

Senior Executive at Wipro (2019-08 – 2020-12)

• Developed ETL workflows and data integration solutions to streamline data movement, enhance processing efficiency, and support analytics requirements.
• Optimized SQL queries and database operations, resulting in faster response times and increased application efficiency.
• Collaborated with cross-functional stakeholders to resolve production issues, stabilize application performance, and ensure successful release delivery within project timelines.

Master of Computer Applications in MCA – Indira Gandhi Delhi Technical University for Women (IGDTUW) (2021-01 – 2023-01)