
Vicky
Data Engineer
Talent
bangaloreMember since June 14, 2026
AwsSnowflake DeveloperData EngineerPythonSQLPL-SQLBashS3GlueEMRAthenaIAMSTSStep FunctionsGitHub ActionsApache SparkPySparkAWS Glue ETLParquetDelta LakeMedallion ArchitectureSnowflakeSnowpipeExternal StageStar Schemadbt Coredbt testsdbt_utilsApache AirflowFileSensor
·
Hire this person
Send a job offer directly to this candidate
AwsSnowflake DeveloperData EngineerPythonSQL
About
Data Engineer with 4+ years of experience building and optimizing production-grade ETL/ELT pipelines in healthcare analytics at IQVIA. Hands-on expertise across Python, PySpark, SQL/PL-SQL, AWS (S3, Glue, EMR, Athena), Snowflake, dbt, and Apache Airflow.
Delivered Medallion
Lakehouse pipelines at 200M+ records/day scale with 99.9% SLA. AWS Certified Data Engineer – Associate and SnowPro Core certified.
Experience
Data Engineer - IQVIA - Bangalore, India
(2022-03 - 2026-04)
- Built and maintained PySpark-on-EMR and AWS Glue ETL pipelines ingesting 200M+ healthcare records daily from S3-landed flat files and Oracle sources into a Medallion Lakehouse (Bronze → Silver → Gold), maintaining 99.9% SLA adherence across all analytical and reporting workloads
- Optimized 20+ complex SQL queries across Snowflake and Oracle PL-SQL for reporting and reconciliation pipelines — applying partitioning, caching, and query-plan analysis — reducing average execution time by ~45% and eliminating recurring weekly SLA breaches
- Built Python-based data validation and reconciliation framework from scratch, enforcing schema, nulls, uniqueness, and referential integrity across pipelines using dbt tests and custom DQ checks — cutting data discrepancy incidents by 60% and reducing manual intervention by 8+ hours/week
- Designed data ingestion frameworks processing structured and unstructured datasets from S3, routing failed records to Dead Letter Queues (DLQ) for safe, idempotent reprocessing — achieving 99.5% pipeline reliability
- Built and maintained dbt staging and mart models (Star Schema) on Snowflake, including surrogate key generation, incremental materialization, and automated dbt tests — enabling self-serve analytics for data scientists and BI teams
- Diagnosed and resolved recurring pipeline failures through structured root-cause analysis, reducing P1 production incidents by 70% over 6 months via Airflow DAG-level retry logic, alerting, and proactive monitoring
- Automated deployment workflows using Git and GitHub Actions CI/CD pipelines, reducing release cycle time from 3 days to same-day deployments and eliminating manual handoff errors
- Designed and deployed interactive Power BI dashboards integrated with Snowflake, enabling senior leadership to track KPIs in real time — supporting 5+ reporting dashboards consumed across the organization
- Received Spotlight Award for automating a manual data analysis process end-to-end, saving ~12 hours/week of analyst effort
Education
Bachelor of Engineering - Computer Science and Engineering - Mahendra College of Engineering (2021)
Skills
Data Engineer
