Aditya Singh

Developed and maintained end-to-end data pipelines using Azure Databricks, PySpark, and SQL, following the Bronze → Silver → Gold → Diamond → Platinum architecture. Handled data landing and ingestion into the Bronze layer, including raw file processing, schema validation, incremental loading, and source-to-Bronze data movement.

Built

Silver and Gold transformations using PySpark and SQL according to business transformation logic.

Data Engineer - Tredence Analytics - Pune

(2025-03)

• Developed and maintained end-to-end data pipelines using Azure Databricks, PySpark, and SQL, following the Bronze → Silver → Gold → Diamond → Platinum architecture.
• Handled data landing and ingestion into the Bronze layer, including raw file processing, schema validation, incremental loading, and source-to-Bronze data movement.
• Built Silver and Gold transformations using PySpark and SQL according to business transformation logic.
• Developed curated diamond and platinum datasets and created Platinum views for downstream reporting and analytical consumption.
• Designed and automated Databricks Workflows with scheduling, monitoring, and failure handling for reliable pipeline execution.
• Consolidated multiple Python notebooks into a single optimized Databricks notebook, improving maintainability and execution.
• Achieved 96% reduction in pipeline runtime, reducing execution time from 2–8 hours to 4–18 minutes through code optimization, parallel processing, and resource tuning.
• Processed and maintained 4 of 9 datasets, ensuring accurate transformations, validation, and data quality.
• Integrated file-upload triggers with Azure Data Factory (ADF) to automate pipeline execution and eliminate manual intervention.
• Automated scheduling, execution, monitoring, and dependency management in ADF, improving reliability and reducing operational overhead.

Data Engineer - Cognizant - Pune

(2023-09 - 2025-03)

• Ingested and integrated retail data from multiple sources into Snowflake for streamlined analysis and reporting.
• Developed complex ETL pipelines using Informatica, improving data accuracy by 37%.
• Built and managed IICS Cloud pipelines and jobs using Snowflake and IICS, improving data reliability by 20%.
• Monitored and resolved job failures, ensuring reliable data transfer from Amazon S3 to Snowflake.
• Collaborated in an Agile team of 5 data engineers and DBAs, participating in peer reviews and maintaining coding standards.

Data Engineer Intern - Cognizant - Pune, MH

(2023-04 - 2023-07)

• Collaborated on designing AWS-based data engineering solutions.
• Gained hands-on experience with Amazon S3, AWS Glue, Lambda, and Redshift.
• Contributed to reliable and scalable data processing architectures.

BE - Electronic Engineering - Shri Ramdeobaba College of Engineering (2019 - 2023) Nagpur, MH

Higher Secondary Education - Saint George College (2018 - 2019) Nagpur, MH