Gnanaprasad Kambham

Azure Data Engineer with 4+ years of experience in designing, developing, and optimizing scalable ETL/ELT pipelines and cloud data solutions. Expertise in Azure Data Factory, Azure Databricks, PySpark, Spark SQL, ADLS, Azure SQL, and CI/CD. Strong experience handling large datasets, building validation frameworks, and delivering analytics in Agile environments.

Azure Data Engineer at Softexer Pvt Ltd (2022-03 – Present)

Client: Marathon Petroleum (Energy Sector)

• Designed and optimized end-to-end data pipelines using Azure Data Factory and Azure Databricks.
• Developed ETL transformations using PySpark and Spark SQL for large datasets.
• Built data validation, logging, and automated monitoring frameworks.
• Automated scheduled and real-time data refresh workflows for analytics teams.
• Integrated ADLS, Azure SQL, Key Vault, and Azure Functions for secure data workflows.
• Built CI/CD pipelines using GitHub and Azure DevOps for deploying data pipelines.
• Prepared optimized datasets for Power BI dashboards and analytical needs.
• Worked with Lakehouse architecture (Bronze, Silver, Gold layers).
• Implemented CDC and incremental load mechanisms using watermarks.
• Created scheduled, event-based, and tumbling window triggers in ADF.

Bachelor of Science in BSC – Yogi Vemana University (2022)