Nadeem Theba

Data Engineer with hands-on experience designing, building, and optimizing production ETL/ELT pipelines, analytical lakehouses, and cloud data platforms on AWS. Specialized in Python (PySpark, Pandas), Advanced SQL, dbt Core, and Apache Airflow. Demonstrated success processing enterprise batch datasets (~500GB+ scale) and real-time streams (50,000 events/sec), building CDC streaming pipelines (Kafka, Debezium, sub-10ms Redis retrieval), eliminating PySpark join memory skew via key-salting, and deploying automated schema contracts (Pandera) to maintain a 99.9% data freshness SLA.

Data Engineer Consultant at Kaival Technologies (2023-10 – Present)

• Optimized high-volume enterprise batch pipelines (~500GB) using DuckDB out-of-core vectorized processing, reducing peak RAM from 32GB+ to <4GB and cutting cloud compute costs by 25%.
• Engineered data validation gates using dbt tests and Pandera schema contracts, routing malformed records to Dead Letter Queues (DLQ) and maintaining 99.9% data freshness SLA.
• Automated legacy data ingestion by engineering modular Python ETL pipelines, migrating 250k+ relational records weekly into cloud storage and eliminating 12 hours of manual operational overhead.
• Executed cloud migration of enterprise datasets to AWS S3 and Snowflake data lakehouses, refactoring batch pipelines and optimizing complex SQL joins across 50M+ rows to reduce query execution latency by 65%.

MSc in Data Science & Analytics with Advanced Research – University of Hertfordshire (2021 – 2023)

BTech in Computer Science – Marwadi University (2016 – 2020)