Kalyan Borra

Data Engineer with 4 years of experience designing and developing scalable ETL/ELT pipelines across cloud and distributed data platforms. Skilled in Python, SQL, PySpark, Spark SQL, Apache Airflow, Azure Data Factory, Azure Databricks, and AWS Glue. Experienced in building data lakehouse architectures, implementing CDC and incremental data processing, optimizing SQL queries, and developing reliable data integration workflows.

Proficient in data quality validation, reconciliation, pipeline monitoring, and CI/CD deployments. Experienced in preparing analytics-ready datasets for enterprise reporting and business intelligence using Azure Synapse Analytics, Power BI, and Tableau.

Data Engineer - CorTracker — Client: Walmart - Fort Worth, TX

(2025-07)

• Maintained 10 ETL/ELT pipelines using Azure Data Factory and Azure Databricks, processing millions of retail records per day to support scalable data integration and downstream analytics.
• Implemented 3 Medallion Lakehouse layers (Bronze, Silver, Gold), organizing raw, cleansed, and curated datasets for reliable reporting.
• Developed PySpark and Spark SQL transformations to standardize data, handle missing values, and remove duplicates, producing consistent analytics-ready datasets.
• Automated data validation and reconciliation checks, targeting a 15% reduction in data quality discrepancies through improved detection of record-count mismatches and inconsistencies.
• Optimized incremental loading, partitioning, and Spark transformations, targeting a 20% reduction in processing time through more efficient data execution.
• Integrated structured and semi-structured data into Azure Data Lake Storage and Azure Synapse Analytics, supporting centralized retail analytics and business intelligence.
• Implemented Azure DevOps CI/CD pipelines, targeting a 20% reduction in manual deployment effort through automated, version-controlled releases.
• Configured Azure Monitor alerts and pipeline monitoring, supporting faster troubleshooting and more reliable pipeline recovery.

Data Engineer - CVS Health - Lowell, MA

(2024-07 - 2025-06)

• Developed and maintained 15 data pipelines using Python, SQL, Spark, and PySpark to process millions of healthcare records for analytics and operational reporting.
• Integrated patient, claims, prescription, and clinical datasets from multiple sources, providing consolidated data for healthcare analysis.
• Developed Apache Airflow DAGs for scheduling, task dependencies, and workflow monitoring, supporting repeatable pipeline execution.
• Built Spark and PySpark transformations to standardize healthcare records, handle missing values, and remove duplicates, improving dataset consistency for downstream analytics.
• Implemented data validation, schema checks, and reconciliation rules, targeting a 15% reduction in data quality discrepancies through systematic source-to-target validation.
• Integrated REST APIs and messaging-based sources into ingestion workflows, expanding data availability for downstream processing.
• Optimized Spark SQL, Hive, and Trino queries, targeting a 20% improvement in query execution time through efficient transformations and query tuning.
• Developed Power BI and Tableau dashboards, providing stakeholders with accessible healthcare operational metrics and reporting insights.

Data Engineer - Prologging Private Limited - Hyderabad, India

(2021-01 - 2022-06)

• Developed Python, SQL, and Apache Airflow ETL workflows to integrate enterprise data sources, supporting centralized data processing and analytics.
• Built AWS Glue ETL jobs and Amazon S3 storage workflows, supporting scalable cloud-based data integration and storage.
• Implemented incremental loading and Change Data Capture (CDC) patterns, targeting a 25% reduction in unnecessary full-data reload processing by processing new and updated records incrementally.
• Optimized SQL queries, stored procedures, and data models, targeting a 20% reduction in query execution time through query tuning and indexing.
• Automated data validation and reconciliation checks, supporting the identification of missing records, duplicates, and source-to-target inconsistencies.
• Integrated Apache Kafka ingestion workflows to support event-driven data processing and downstream system integration.
• Developed dbt-based transformation models, improving the organization and maintainability of analytical data models.
• Configured Apache Airflow task dependencies and schedules, supporting repeatable workflow execution and operational monitoring.

Master of Science - Data Science - Lewis University (2022-08 - 2024-05)

Bachelor of Technology - Civil Engineering - Jawaharlal Nehru Technological University (2018-08 - 2021-03)