Kamal Palagiri

Azure Data Engineer with 4+ years of experience building and optimizing large-scale ETL/ELT pipelines and cloud data platforms on Azure. Delivered 20+ production pipelines using Azure Data Factory, Azure Databricks, PySpark, and Microsoft Fabric, cutting data-processing time by 30% and improving pipeline reliability to 99.5% uptime. Skilled in Medallion architecture, Delta Lake, Synapse Analytics, and CI/CD automation with Azure DevOps.

Azure Data Engineer at AIML Labs Pvt Ltd (2022-04 – Present)

• Designed and delivered 20+ end-to-end ETL/ELT pipelines on Azure Data Factory and Azure Databricks, processing 5 million+ records/week from 4+ source systems into a governed Lakehouse.
• Re-architected batch pipelines into a Medallion (Bronze/Silver/Gold) model, cutting data-refresh latency from 6 hours to 2 hours and reducing downstream data-quality defects by 25%.
• Wrote PySpark transformation logic and complex SQL/stored procedures to implement business rules, improving average pipeline runtime by 35% through partitioning and cluster-sizing optimizations.
• Automated deployment of data workflows using Azure DevOps CI/CD pipelines, reducing manual release effort by 40% and eliminating environment drift.
• Implemented Delta Lake and Parquet-based storage strategies, cutting storage costs by 20% while improving query performance on Synapse Analytics.
• Enforced data governance and access control using RBAC, supporting compliance audits with zero critical findings.
• Partnered with business and analytics teams in an Agile setup (JIRA, Confluence) to translate requirements into production-ready pipelines, consistently meeting sprint commitments.

B.Sc. in Computer Science – Sree Vidyanikethan Degree College (2019-06)