Dataengineer

Senior Data Engineer with 7+ years of experience designing and building scalable batch and real-time data pipelines across AWS, Azure, and GCP. Deep expertise in Apache Spark, Databricks, Kafka, Snowflake, and Python. Proven track record architecting lakehouse solutions with Delta Lake and Apache Iceberg, implementing CDC pipelines, and automating infrastructure using Terraform and CI/CD.

Experienced across high-compliance industries including healthcare and insurance, delivering production-grade platforms that power analytics, ML, and real-time decision-making.

Senior Data Engineer at DBHDS (2026-02 – Present)

Designed and implemented cloud-native data pipelines to ingest, process, and curate behavioral health and healthcare data from multiple source systems into a centralized enterprise data platform.

• Built scalable ETL/ELT workflows using AWS Glue, PySpark, Python, and SQL, enabling daily ingestion and transformation of structured and semi-structured healthcare datasets.
• Implemented Medallion Architecture (Bronze, Silver, Gold) on AWS to standardize data processing, improve data quality, and provide analytics-ready datasets for reporting and downstream applications.
• Developed automated ingestion frameworks for healthcare data exchanges, including HL7 and FHIR-based integrations, ensuring secure and reliable movement of patient and clinical information.
• Designed and optimized Iceberg/Delta Lake tables to support incremental processing, historical tracking, and high-performance analytical queries.
• Built data quality and validation frameworks using PySpark and SQL to perform schema validation, duplicate detection, reconciliation, and business rule enforcement before publishing data to curated layers.
• Leveraged AWS Lambda, EventBridge, and Step Functions to automate data ingestion workflows, orchestrate dependent jobs, and reduce manual operational effort.
• Developed secure API-based integration solutions to exchange healthcare and operational data with external agencies, partner systems, and internal applications.
• Optimized Spark workloads through partitioning, caching, broadcast joins, and query tuning, significantly improving processing performance and reducing cloud resource consumption.
• Collaborated with business analysts, healthcare stakeholders, and architects to gather requirements and translate complex business needs into scalable data engineering solutions.
• Implemented enterprise security controls using IAM, KMS, Lake Formation, and Secrets Manager, ensuring HIPAA-compliant handling of sensitive healthcare data.
• Built monitoring and alerting solutions using CloudWatch, SNS, and Glue job metrics, enabling proactive detection and resolution of production issues.
• Supported AI/ML and advanced analytics initiatives by creating curated datasets and feature-ready data structures for predictive modeling and healthcare insights.
• Participated in Agile ceremonies, code reviews, production support, and release planning while mentoring junior engineers and promoting engineering best practices.
• Created detailed technical documentation, source-to-target mappings, operational runbooks, and data lineage artifacts to support governance, audit, and knowledge transfer activities.

Senior Data Engineer at FM Global Insurance (2024-03 – 2025-12)

• Designed and implemented real-time streaming pipelines using Apache Kafka, Flink, and Spark Structured Streaming to process insurance policy, claims, and risk event data.
• Built Kafka producer/consumer services and ksqlDB/Flink transformations to ingest high-volume policy transactions and enrich underwriting data streams for downstream analytics.
• Architected Azure Databricks streaming pipelines consuming Kafka topics and persisting curated datasets in Azure Data Lake using Delta Lake (Bronze/Silver/Gold) architecture.
• Implemented CDC ingestion pipelines from SQL Server policy databases, publishing incremental changes to Kafka topics with Confluent Schema Registry for schema validation.
• Automated end-to-end workflows via Azure Data Factory and Databricks Jobs with full dependency management, retry logic, and error alerting.
• Embedded DataOps practices via Azure DevOps CI/CD pipelines automating deployment of notebooks, pipelines, and Terraform-managed infrastructure (Kafka connectors, Azure resources).
• Implemented data lineage and pipeline monitoring using Dynatrace and Azure Monitor, detecting anomalies and ensuring SLA compliance.
• Optimized streaming throughput via checkpointing, Kafka partition tuning, and consumer group scaling — reducing data latency from hours to minutes.

Senior Data Engineer at Mayo Clinic (2023-04 – 2024-02)

• Built scalable healthcare data pipelines on AWS (S3, Glue, EMR) to ingest and process large volumes of patient and clinical data from EHR systems in compliance with HIPAA standards.
• Developed PySpark transformation jobs on EMR to cleanse, normalize, and enrich patient encounter, diagnostic, and medical records data stored in Parquet format on S3.
• Implemented real-time healthcare event ingestion using Kafka and AWS Lambda, capturing hospital system events with low latency.
• Developed Apache Airflow DAGs orchestrating clinical data ingestion, transformation, and loading into Snowflake for analytics and patient outcome reporting.
• Built Snowflake data models supporting clinical reporting, patient outcomes analytics, and research data sets for medical research teams.
• Secured healthcare data with AWS IAM policies, KMS encryption, and role-based access controls meeting regulatory compliance requirements.
• Automated infrastructure provisioning using Terraform and CI/CD pipelines, reducing manual deployment effort and ensuring reproducible environments.
• Optimized Spark jobs on EMR by tuning memory allocation, partitioning strategies, and caching — significantly improving processing throughput for large clinical datasets.

Data Engineer at Express Scripts (Cigna) (2021-11 – 2023-03)

• Designed and implemented cloud-based enterprise data lake across AWS S3 and Google Cloud Storage to process large-scale healthcare and pharmacy claims datasets.
• Built scalable ETL pipelines using Apache Airflow, GCP Dataproc, and PySpark to transform structured/semi-structured healthcare data, reducing processing time by improving parallelism.
• Developed streaming ingestion pipelines using GCP Pub/Sub and Kafka to capture real-time prescription transactions and pharmacy claim events from operational systems.
• Migrated legacy Oracle and Netezza data warehouse workloads to BigQuery and Amazon Redshift, improving scalability and reducing query latency.
• Designed incremental CDC pipelines to capture transactional database changes and load them into cloud data lake environments for near-real-time analytics.
• Built Customer Data Platform (CDP) ingestion frameworks integrating transactional, behavioral, and marketing datasets supporting 360-degree customer analytics.
• Integrated Power BI dashboards with BigQuery and Redshift to enable real-time monitoring of prescription trends, pharmacy operations, and engagement metrics.
• Implemented security and governance controls using IAM, KMS encryption, and RBAC to ensure HIPAA compliance across cloud data environments.

Cloud Data Engineer at Aetna (CVS Health) (2020-10 – 2021-07)

Implemented and managed Enterprise Data Lake on AWS, supporting high-volume analytics, processing, and reporting use cases for healthcare insurance data. Developed fine-grained S3 access control security framework.