Varun B

Build and optimize real-time pipelines for high-volume, production-critical systems across financial, healthcare, and telecom domains, with 3+ years of hands-on experience. Operate at 50TB+ scale and process 1M+ events/day across Azure, AWS, and GCP, delivering reliable ETL/ELT and streaming architectures. Fix broken pipelines, reduce latency, and deliver platforms teams trust for compliance, analytics, and operational decision-making.

Azure Data Engineer - Navy Federal Credit Union - Vienna, VA

(2025-05)

• Rebuilt a broken ETL/ELT pipeline in Azure Data Factory (ADF) and Synapse Analytics by offloading heavy SQL transformations to dedicated Synapse pools and restructuring ingestion stages. Reduced a 4-hour delay to less than 2.6 hours (35%) and restored reliable data delivery to compliance and trading teams.
• Designed a PySpark + Databricks pipeline that supports fraud analytics using Medallion architecture, enabling auditability across Bronze/Silver/Gold layers and improving data readiness for model consumption, contributing to a 25% increase in detection accuracy.
• Owned ingestion of 50TB+ financial data across 15+ sources using ADF, NiFi, and Talend. Implemented validation checks and lineage tracking to manage schema evolution and prevent upstream changes from impacting compliance reporting.
• Optimized SQL transformations in Synapse Analytics using layered architecture, reducing transformation runtime by 30% and improving regulatory report turnaround by 40%.
• Orchestrated event-driven pipelines using Kafka and Airflow, maintaining 99.9% pipeline availability. Automated deployments with Azure DevOps, Docker, and Kubernetes, reducing release time by 45%.

AWS Data Engineer - Owens & Minor - Mechanicsville, VA

(2024-10 - 2025-04)

• Stabilized a failing healthcare streaming pipeline using PySpark, Kinesis, Lambda, and Kafka. Resolved frequent job failures across a 1M+ events/day pipeline, improving data accuracy from ∼92% to 98% and reducing failure rates by 35%.
• Automated ingestion and processing of medical order and inventory data using AWS Glue, Lambda, and S3, reducing manual ETL effort by 60% and improving data consistency across supply chain workflows.
• Built a scalable data lakehouse on Redshift and Snowflake with Delta Lake, enabling ACID transactions and schema evolution, and improving query performance by 50% across analytics workloads.
• Handled high-volume event streams using PySpark and Spark Structured Streaming. Built CI/CD workflows with GitLab CI and Jenkins, reducing release failures by 35% and shortening deployment cycles by 2 days.

Data Engineer - Comcast - Chennai, India

(2022-03 - 2024-07)

• Redesigned GCP analytics workflows using BigQuery, Dataflow, and Pub/Sub with star schema modeling. Reduced query latency from ∼8 seconds to under 1 second, enabling analytics on 100M+ daily telecom events.
• Maintained telecom ingestion pipelines using Kafka, PySpark, and Airflow, integrating 10+ data sources across CRM, usage, and operations systems and reducing pipeline build time by 40%.
• Implemented data governance practices using Apache Atlas and Collibra, improving data visibility, lineage tracking, and regulatory readiness across distributed datasets.
• Enabled analytics teams by delivering curated datasets and KPI layers for churn and network performance, contributing to a 12% improvement in customer retention.
• Automated churn analysis pipelines using SQL, PySpark, Pandas - reducing manual reporting effort and cutting analysis time by ∼30%.

M.S. - Computer Science - Virginia Commonwealth University (2024-08 - 2025-12)