Kelvin Danyal

Senior Data Engineer with 12+ years designing and managing lakehouse platforms and real-time streaming pipelines. Expert in Spark, Flink, Kafka, Delta Lake, Iceberg, Snowflake, and Databricks. Skilled in AI/ML pipelines, LLM and RAG workflows, vector databases, and cloud-native healthcare data platforms. Proven record processing billions of events daily with 99.9%+ uptime.

Senior Data Engineer at Techinoxsystems (2023-04 – Present)

• Lead lakehouse platform supporting 2B+ events/day across 3 AWS regions for 50+ analysts and 10 ML models.
• Built streaming pipelines (Kinesis + Lambda + Iceberg) reducing batch lag from 8h to 2min.
• Developed 350 dbt models with automated freshness checks.
• Engineered AI-ready feature stores for RAG and LLM pipelines.
• Automated PII detection using AWS Macie and Python classifiers at 99.7% accuracy for HIPAA compliance.
• Optimized Snowflake warehouses, saving $45k/month.
• Implemented real-time anomaly detection pipelines using Spark Structured Streaming.
• Introduced automated model retraining pipelines with Airflow and MLflow.
• Designed multi-region failover architecture ensuring zero downtime.

Senior Data Engineer at Centric Consulting (2020-06 – 2023-03)

• Delivered streaming and batch pipelines for enterprise clients across retail, insurance, and manufacturing.
• Built customer-360 warehouse integrating Shopify, Salesforce, and Google Analytics for 30 analysts.
• Reusable Airflow DAG templates reduced pipeline development from 2 weeks to 3 days.
• Implemented automated data-quality monitoring, catching 95% of schema issues pre-production.
• Developed real-time event-driven ETL pipelines using Kafka and Spark Streaming.
• Introduced feature stores and metadata-driven pipeline templates reducing new project setup by 50%.

Senior Data Engineer at Flatiron Health (2018-09 – 2020-05)

• Maintained ETL pipelines for oncology real-world data from 280 community practices.
• Built EHR data abstraction pipelines using Spark and NLP for genomic, pathology, and clinical notes.
• Engineered Delta Lake and Databricks lakehouse for structured and unstructured clinical datasets.
• Mentored junior engineers; reduced research-ready dataset prep from 6 weeks to 5 days.
• Designed RAG pipelines for clinical NLP, improving document retrieval 15x.

Data Engineer, Early Career at Various Clients (2014-01 – 2016-12)

• Built analytics warehouses, ETL pipelines, and streaming data solutions for healthcare and enterprise clients.
• Worked on projects involving data warehousing, ETL processes, and real-time data streaming.
• Collaborated with cross-functional teams to deliver scalable data solutions.

B.S. in Computer Science – University at Buffalo (2014)