Shiva Kumar

Azure Data Engineer with 4.2+ years of experience designing, building, and deploying scalable, production-grade data pipelines and cloud-native data platforms, primarily on Microsoft Azure with hands-on exposure to AWS and Snowflake. Proficient in Azure Data Factory, Azure Databricks, PySpark, SQL, and Delta Lake, with expertise architecting Medallion-based data lakes (Bronze/Silver/Gold), metadata-driven ingestion frameworks, and incremental/CDC-based ETL pipelines. Extending batch architectures with Microsoft Fabric (OneLake, Lakehouse) and near-real-time streaming ingestion via Azure Event Hubs and Spark Structured Streaming.

Delivered measurable engineering impact, including 30-40% ETL performance gains and 25-35% reductions in downtime and manual effort. Experienced across healthcare, insurance, and retail domains within Agile/Scrum teams, with strong command of data quality, pipeline monitoring, security (RBAC, Key Vault), and CI/CD via Azure DevOps.

Data Engineer at Axisweb soft technologies Pvt Ltd (2022-05 – 2026-07)

• Architected and deployed scalable, cloud-native ETL pipelines using Azure Data Factory and Azure Databricks to migrate on-premises workloads to Azure, improving reliability and maintainability of enterprise data processing.
• Improved data pipeline performance by 30% by implementing partitioning, caching, and incremental load strategies across Azure Data Factory and Databricks workflows, reducing compute cost and processing time.
• Engineered a metadata-driven, parameterized ingestion framework in Azure Data Factory, cutting manual intervention and accelerating onboarding of new data sources for faster time-to-insight.
• Extended existing ADF/Databricks batch pipelines with Microsoft Fabric (OneLake, Lakehouse) and near-real-time streaming ingestion via Azure Event Hubs and Spark Structured Streaming to advance the platform toward a modern lakehouse architecture.
• Managed CI/CD pipelines through Azure DevOps and version control via Git, standardizing deployment of data engineering workflows and reducing release risk across sprint cycles.
• Partnered with data analysts, architects, and business stakeholders across Agile/Scrum ceremonies (sprint planning, stand-ups, backlog grooming, retrospectives) to deliver iterative, production-ready data solutions.

Bachelor's Degree – Silver Jubilee Government College (2022)