
John White
Senior Data Engineer
Hire this person
Send a job offer directly to this candidate
About
Senior Data Engineer with 10+ years of experience building cloud data platforms, modern data warehouses, and AI-ready Lakehouse solutions across retail/eCommerce, insurance, and healthcare. Expertise in Databricks, Snowflake, Spark, Python, SQL, and AWS, with a strong background in migrating legacy data platforms, developing scalable data pipelines, and delivering trusted data products for analytics, machine learning, and business intelligence.
Experience
Senior AI Data Platform Engineer at Goldbelly (2024-02 – Present)
- Architected an AWS-based Databricks–Snowflake Lakehouse using Delta Lake Medallion Architecture, unifying customer, order, merchant, payment, and fulfillment data into governed data products that processed 8+ TB daily, reduced data delivery latency by 32%, and powered enterprise analytics, ML, and executive reporting.
- Engineered an AI-powered Recommendation Platform using PySpark, Databricks, Feature Store, Vector Search, and RAG/LLM frameworks, creating reusable governed feature pipelines that supported 15M+ personalized recommendations daily, reduced feature generation time by 18%, and standardized feature computation and lineage.
- Established resilient batch and real-time data pipelines with Kafka, Spark Structured Streaming, Airflow, CDC, Delta Lake, and dbt, processing 500M+ events daily while reducing end-to-end latency by 65% for AI inference, operational analytics, and BI.
- Integrated customer, recipient, subscription, loyalty, and behavioral data into governed Gold-layer datasets with identity resolution, enabling advanced segmentation, lifecycle analytics, fraud detection, and personalized marketing.
- Strengthened Lakehouse security by implementing PII masking, row-level security, and Unity Catalog access policies, improving governance and reducing unauthorized access incidents by 40%.
- Orchestrated governed ELT ingestion using Unity Catalog, Snowflake, Delta Lake, Fivetran, Matillion, and Amazon S3, improving data lineage, governance, and secure self-service data sharing across engineering and business teams.
- Optimized Databricks clusters and Snowflake warehouses through right-sizing, query tuning, and governance controls, reducing compute costs by 30% while maintaining analytics and AI SLAs.
- Automated cloud infrastructure and CI/CD workflows using Terraform, Kubernetes, Docker, GitHub Actions, and AWS, reducing deployment time by 42% and increasing release frequency from weekly to daily.
- Partnered with Data Scientists, ML Engineers, Product Managers, and Software Engineers to deliver AI-powered data solutions, improve distributed Spark and Snowflake workloads, and mentor junior engineers on Lakehouse architecture and performance tuning.
Senior Consultant | Data & Intelligence at Slalom (2022-01 – 2024-01)
- Developed scalable ETL/ELT pipelines for Slalom's client, CRICO, modernizing the Candello healthcare risk intelligence platform using Python, SQL, Snowflake, dbt, AWS (Lambda, S3), and REST APIs to integrate medical malpractice claims, provider, patient safety, policy, and financial data from 12+ enterprise healthcare systems.
- Built automated ELT pipelines with Fivetran, Matillion, Snowflake, AWS Lambda, and Amazon S3, reducing manual integration effort by 70% while improving pipeline reliability and monitoring through Amazon CloudWatch.
- Enhanced Snowflake warehouse performance using clustering keys, materialized views, query profiling, dbt incremental models, and SQL optimization, reducing complex reporting query execution times by 45%.
- Created reusable dbt transformation models with Jinja macros, snapshots, incremental models, lineage tracking, and data quality tests, standardizing business logic for claims, provider analytics, patient safety reporting, and executive dashboards.
- Implemented enterprise data quality and governance processes using Python, SQL, dbt tests, Snowflake Tasks, stored procedures, and reconciliation workflows to improve healthcare data accuracy and integrity.
- Delivered interactive Power BI dashboards using Snowflake, semantic models, DAX, Power Query (M), and row-level security, providing real-time visibility into claims, patient safety, provider risk, litigation costs, and operational KPIs for 30+ stakeholders.
- Collaborated with business analysts, healthcare SMEs, QA engineers, and application teams in Agile environments, supporting GitHub Actions CI/CD, code reviews, automated testing, and AWS-based pipeline deployments.
Senior Data Engineer / Senior Business Analyst at American National (2016-03 – 2021-11)
- Modernized an enterprise insurance data warehouse by migrating policy, claims, billing, customer, agent, commission, finance, and actuarial data from 8+ legacy SQL Server and Oracle systems to an AWS-based analytics platform, supporting 500+ business users with a centralized source of trusted enterprise data.
- Designed and developed scalable ETL pipelines using SSIS, Informatica PowerCenter, SQL, T-SQL, and Python, implementing incremental loading, metadata-driven workflows, audit logging, and restartable batch processing that reduced nightly batch processing time by approximately 45%.
- Built dimensional warehouse models consisting of 20+ fact and dimension tables, implementing SCD Type 2, customer identity resolution, duplicate detection, and late-arriving data processing to support 5+ years of historical policy, claims, and financial analytics.
- Developed enterprise data quality and reconciliation frameworks using AWS Glue, SQL, and Python, validating millions of insurance records while significantly reducing manual reconciliation efforts and improving the reliability of financial, regulatory, and operational reporting.
- Optimized ETL and warehouse performance through set-based SQL transformations, query tuning, indexing, partitioning, parallel processing, and Amazon Redshift optimization, reducing report refresh times from several hours to under one hour while consistently meeting nightly processing SLAs.
- Collaborated with underwriting, claims, finance, actuarial, compliance, and BI teams to gather requirements, define enterprise KPIs, create source-to-target mappings, coordinate UAT, and deliver 50+ Power BI, Tableau, and SSRS dashboards and reports, enabling self-service analytics and faster business decision-making.
Education
Bachelor of Science in Mathematics – University of Houston (2010-08 – 2015-08)