Data Engineer at Guardant Health Business Solutions Pvt Ltd (2025-06 – Present)
Project: Custom Reports
- Built an automated email alerting solution for missing client data, reducing report data gaps by 90% and enabling triage within 30 minutes of detection.
- Orchestrated a fault-tolerant pipeline using Airflow and AWS Batch across S3/Athena to validate daily ingestions, eliminating manual checks (100% reduction) and maintaining ≥99% SLA adherence.
- Developed AWS Lambda functions for real-time schema and arrival checks on S3, triggering automated remediation workflows and stakeholder notifications.
- Designed AWS Glue (Pyspark) reporting pipelines that improved report freshness by 94% (from 3 hours to 10 minutes) while reducing operational costs by 90%.
- Implemented Athena-backed data quality rules and partitioning strategies, slashing query costs by 50% and boosting performance by 30%.
- Designed and deployed an AI-powered tagging system using AWS Bedrock (LLM) to process CAPA and NCR event data, automatically generating summary and repeat tags for dashboard insights.
- Built the end-to-end solution independently, including prompt design, model integration, pipeline orchestration, and production deployment, improving event analysis efficiency and enabling faster decision-making.
Data Engineer at Teladoc Health Business Solutions Pvt Ltd (2024-01 – 2025-06)
Project: Atomic Datawarehouse
- Spearheaded the successful migration of ETL pipelines from legacy Talend systems to Databricks, managing the full lifecycle from initial analysis to production deployment.
- Independently reverse-engineered and rebuilt 50+ Talend workflows into optimized PySpark-based Databricks notebooks, achieving a 300% increase in data processing speed.
- Engineered complex transformation logic to normalize and enrich healthcare datasets, ensuring strict adherence to medical business rules and industry compliance standards.
- Collaborated with cross-functional data stakeholders to define business requirements, testing protocols, and delivery milestones, ensuring seamless project alignment.
- Demonstrated strong problem-solving capabilities by delivering the high-impact migration project single-handedly within the designated deadline.
Data Engineer at BP Business Solutions Pvt Ltd (2021-06 – 2024-01)
Project: Cognite Data Fusion
- Planned, Orchestrated, and architected the pipeline for data flow (~100 GB) from BP to Cognite for large scale contextualization of data using cloud based distributed data processing applications.
- Created and supported the data transformation notebooks in Azure Databricks using Pyspark, Spark – SQL.
- Used big data technology tools to interact with Data Lake for Data Transformation. Performed ETL on disparate data sources like Wells, Wellbores, casing etc. for better visualization to Business.
Data Engineer at UBS Business Solutions Pvt Ltd (2018-07 – 2021-06)
Project: Confucius under IB – Data Analytics
- Developed data pipeline for the derivatives data using Azure Data Factory.
- Lead the migration from on-prem to Azure cloud, data volume 100GBs.
- Developed and orchestrated the Data Ingestion and Transformation pipelines using Azure Databricks.
