Karthik kumar Tanniru

Data Engineer at Sagitta Consulting Solutions India Pvt Ltd (2023-04 – Present)

Designed and maintained Azure Data Factory pipelines ingesting structured and semi-structured data from flat files, REST APIs, and on-premises systems into ADLS Gen2, handling GBs of daily data with scheduled and incremental load patterns — reducing manual data delivery effort by ∼60%.
Built PySpark transformation layers in Azure Databricks for cleansing, deduplication, and aggregations; cut average data preparation time by 35% and enabled same-day reporting for 3 downstream analytics teams.
Architected a Medallion (Bronze/Silver/Gold) data model in Azure Synapse Analytics, reducing redundant data processing and supporting dashboards for 5+ business stakeholders.
Improved pipeline reliability from ∼85% to 98%+ by implementing structured monitoring with Azure Monitor and Log Analytics, configuring alerts on failure thresholds and documenting resolution playbooks.
Standardized pipeline deployments using Azure DevOps CI/CD with Git-integrated ADF publishing, eliminating manual environment errors across Dev, QA, and Prod.
Established automated data quality checks using PySpark assertions on row counts, null rates, and schema conformance — catching 90%+ of upstream data issues before they reached reporting layers.
Collaborated with business analysts and BI teams to translate data requirements into pipeline specifications, ensuring consistent data definitions and minimising rework cycles.

About