Ayushi Gupta

Azure Data Engineer with 3+ years of experience building scalable data pipelines using Azure Data Factory, Azure Databricks, and PySpark. Strong expertise in designing ETL/ELT workflows, data modeling, and processing large-scale datasets in cloud environments (ADLS Gen2, Delta Lake). Proven track record of improving performance, optimizing data pipelines, and delivering analytics-ready datasets for enterprise applications and business reporting.

Senior Software Engineer at Coforge (2025-10 – Present)

Built scalable Azure-based data pipelines using Azure Data Factory and Azure Databricks processing financial data

• Built scalable Azure-based data pipelines using Azure Data Factory and Azure Databricks (PySpark) processing ~70 GB/day of financial data through daily batch and incremental pipelines.
• Designed and optimized ETL workflows for high-volume structured data, improving system efficiency and reliability.
• Optimized Azure SQL and Databricks workloads, improving query performance by 35% using indexing, partitioning, and execution plan tuning.
• Migrated legacy SQL-based transformations to distributed PySpark workflows, significantly reducing processing time.
• Optimized Spark jobs using partitioning, caching, and broadcast joins to improve processing efficiency.
• Implemented pipeline monitoring, logging, and error-handling mechanisms in ADF and Databricks for failure tracking and reliability.
• Performed data validation and reconciliation checks to ensure high data quality across downstream systems.
• Reduced production incidents by 25% through proactive monitoring and root cause analysis.
• Enabled faster reporting and analytics by delivering clean, reliable, and structured datasets for business stakeholders.
• Collaborated with cross-functional teams including analysts and business users to deliver data-driven solutions.

Software Engineer at Coforge (2023-06 – 2025-10)

Developed ETL pipelines and data ingestion workflows for enterprise financial data processing

• Developed ETL pipelines using SQL and Python for enterprise financial data processing.
• Built data ingestion and transformation workflows supporting batch and incremental processing systems.
• Improved query performance by 50% through SQL tuning, indexing strategies, and execution plan optimization.
• Designed and maintained database objects including tables, views, stored procedures, and materialized views.
• Supported data migration initiatives from on-premise systems to modern cloud-based platforms.
• Automated validation and reporting processes using Python scripts, improving operational efficiency.

B.Tech in Computer Science & Engineering – Noida Institute of Engineering and Technology (2019-06 – 2023-06)