Nikhil Jemili

Data Engineer with 5 years of experience in designing, developing, and supporting scalable batch and streaming data pipelines, cloud data lakes, warehouses, and analytics-ready datasets. Experienced with Python, SQL, PySpark, Apache Spark, AWS, Databricks, Airflow, Snowflake, dbt, Kafka, and Delta Lake, with a focus on improving pipeline reliability, processing performance, data quality, and reporting availability. Proven ability to translate business requirements into maintainable data solutions while supporting production operations and downstream analytics across multiple business domains.

Data Engineer - UnitedHealth - Dallas, TX

(2024-08)

• Designed batch and incremental pipelines using Python, PySpark, Azure Databricks, Delta Live Tables, and Databricks Workflows, processing 18 million records daily across governed analytics datasets.
• Designed Azure ingestion workflows using Azure Data Factory and ADLS Gen2, integrating PostgreSQL, API, and file-based sources into standardized Delta Lake storage layers.
• Developed Kafka streaming pipelines with Apache Spark and Change Data Capture, reducing operational data latency from 10 minutes to under 3 minutes for downstream consumers.
• Implemented Snowflake dimensional models with dbt, star and snowflake schemas, SCD Type 2, and incremental models across 42 analytics tables supporting reporting workloads.
• Automated quality controls using Great Expectations, dbt tests, source-to-target reconciliation, schema validation, and lineage checks, strengthening release confidence across business-critical production datasets.
• Established GitHub Actions CI/CD with Git, Docker, Terraform, Bash scripting, and AWS IAM, reducing deployment effort from 5.5 hours to 2 hours across data releases.
• Led production troubleshooting through Amazon CloudWatch and Databricks Monitoring, lowering average pipeline recovery time from 64 to 38 minutes through improved alerting and diagnostics.
• Optimized PySpark and Delta Lake workloads in Azure Databricks through partitioning, file compaction, and incremental processing, accelerating recurring pipeline runtimes and resource efficiency across priority workflows.
• Enhanced analytics data availability by tuning Databricks Workflows and Snowflake incremental loads, improving daily dataset readiness and reporting consistency for analysts and downstream consumers.
• Coordinated AWS and Azure platform changes with analysts, architects, and engineers, translating requirements into governed pipelines, documented dependencies, and reliable operational procedures.

Data Engineer - Sundaram Finance - India

(2021-11 - 2023-07)

• Built 18 ETL and ELT pipelines with Python, SQL, AWS Glue, Airflow, and S3, ingesting 8 million daily records from varied sources.
• Migrated 12 operational datasets into Amazon Redshift and S3, redesigning transformations and Parquet layouts to reduce reporting preparation time by 24%.
• Optimized PySpark workloads on Amazon EMR through partitioning, predicate pushdown, caching, and compaction, cutting average batch runtime by 23% across recurring jobs.
• Created four Amazon Kinesis event pipelines with AWS Lambda and Step Functions, reducing source-to-warehouse latency from 18 minutes to 7 minutes.
• Configured DynamoDB ingestion patterns and AWS IAM policies for event-driven workloads, improving secure data exchange and simplifying service permissions across engineering teams.
• Introduced Great Expectations, SQL reconciliation, schema validation, and CloudWatch alerts, reducing downstream data corrections by 21% across critical production pipelines.
• Supported Git releases, Docker environments, Terraform-managed AWS infrastructure, and Bash deployment scripts while documenting lineage, dependencies, recovery procedures, and operational handoffs.

Associate Data Engineer - Coforge - India

(2020-09 - 2021-10)

• Created eight scheduled ingestion jobs using Python and SQL to move PostgreSQL and flat-file data into S3, processing 3 million records weekly.
• Developed SQL transformations for 12 Amazon Redshift reporting tables, applying dimensional modeling and star schemas while saving seven hours per reporting cycle.
• Configured Airflow DAGs with dependencies, retries, and notifications for six recurring workflows, increasing successful unattended execution from 91% to 97%.
• Engineered PySpark cleansing and deduplication routines for Parquet datasets in S3, reducing duplicate and malformed records by 18% across five source feeds.
• Validated nulls, row counts, schemas, and source-to-target totals using Python and SQL across 20 tables, reducing release-related corrections by 16%.
• Maintained Git repositories, Docker development environments, and CloudWatch logs while troubleshooting failed jobs, documenting schemas, and supporting controlled deployments across ingestion processes.
• Assisted senior engineers with AWS Glue jobs, Lambda functions, Redshift loading, and IAM permissions, building foundational experience in cloud pipeline operations and security.

Master of Science - Computer Science and Engineering - State University of New York at Buffalo (2023-08 - 2024-12)

Bachelor of Computer Applications - Computer Applications - Christ University (2018-07 - 2021-05)