Shiva

Highly accomplished, certified Azure Data Engineer with 5 years of enterprise experience engineering robust ETL/ELT pipelines and scalable cloud data platforms. Recognized for expertise in Azure Data Factory (ADF), Azure Databricks, PySpark, and Microsoft Fabric to transform complex datasets into high-value analytical models. Expert in implementing unified data governance, optimizing medallion architectures, and automating testing frameworks to maximize platform reliability.

Data Engineer at Fidelity Consultancy (Client) (2024-05 – 2026-08)

Environment: ADF, Databricks, Microsoft Fabric, Synapse Analytics, ADLS Gen2, Delta Lake, Python, PySpark, SQL, Unity Catalog, Azure Purview, Azure DevOps

• Architected and optimized high-throughput ETL/ELT pipelines using Azure Data Factory and Synapse Analytics, ensuring reliable downstream analytical processing.
• Engineered multi-node PySpark transformations within Databricks, optimizing cluster memory management to smoothly process terabyte-scale enterprise workloads.
• Developed metadata-driven pipelines utilizing Delta Lake and ADLS Gen2, completely standardizing ingestion patterns and reinforcing data lineage visibility.
• Built scalable data platform solutions deploying Microsoft Fabric lakehouses and medallion architecture patterns to accelerate corporate reporting speeds.
• Programmed custom Python frameworks for automated data modeling and validation, eliminating repetitive operational tasks and manual development efforts.
• Enforced enterprise data governance profiles using Unity Catalog and Azure Purview, securing cloud environments while deploying automated CI/CD code via Azure DevOps.

Data Engineer at Origin Energy (Client) (2021-04 – 2022-10)

Environment: Airflow, AWS S3, Redshift, Azure, Databricks, Synapse, Python, PySpark, Delta Lake, ADLS Gen2, Power BI, Azure DevOps

• Deployed robust hybrid cloud data-processing workflows, pairing Apache Airflow orchestration seamlessly with AWS S3, Redshift, and core Azure storage tiers.
• Partnered with senior business stakeholders to translate complex corporate requirements into concrete, end-to-end cloud pipeline specifications.
• Executed high-performance PySpark configurations to optimize data transformations across massive, production-grade enterprise structures.
• Automated recurring unit testing strategies and staging migrations by designing modular, reusable Python script libraries.
• Structured dynamic metadata frameworks with Delta Lake and Azure Purview to guarantee data asset validation and strict operational compliance.

BI & Data Engineer at BDC Travel (Client) (2020-11 – 2021-04)

Environment: SSIS, SSAS, ADF, Databricks, Synapse, Python, PySpark, ADLS Gen2, Azure SQL Database, SAP BW/BO, Power BI

• Spearheaded legacy platform modernization by seamlessly migrating on-premises SSIS/SSAS infrastructure into Azure Data Factory and Databricks clusters.
• Collaborated on enterprise-wide SAP BW integrations, bridging core analytical layers to produce highly accurate, unified corporate reporting streams.
• Formulated advanced Python and PySpark script operations to transform disparate relational databases into organized data lake structures.
• Designed semantic data models using SSAS cubes and configured structured data flows to ensure optimal dashboard loading performance.
• Developed comprehensive business intelligence dashboards in Power BI and SAP BusinessObjects to deliver self-service analytics to operational stakeholders.

BI & Data Engineer at DBS Bank (Client) (2019-10 – 2020-11)

Environment: SSIS, SSAS, SSRS, SQL, Alteryx, SAP, Python, Power BI, Tableau

• Developed enterprise-grade ETL systems utilizing SSIS, SQL, and Alteryx to aggregate critical financial transactional fields out of central SAP systems.
• Constructed highly interactive Power BI and Tableau dashboards connected directly to SSAS models to foster self-service executive discovery.
• Automated data validation routines using Python scripting to run post-ETL consistency checks across legacy and modern database schemas.
• Successfully migrated static, legacy SSRS paginated reports into interactive Power BI report environments, expanding data discovery capabilities for regional teams.

MSc Management with Extended Professional Practice – Coventry University London Campus (2022 – 2024)

Bachelor's in Electronics & Communication Engineering – JNTU, Kakinada (2016 – 2020)