Dillibabu M

Engineered end-to-end data pipeline for loan collection campaigns using Spark & Airflow, integrating engagement data across multiple sources, performing multi-layer reconciliation and data validation, and delivering real-time campaign performance metrics and reports.

Data Engineering - SPX Collections Pipeline at Spocto (2026-01)

• Designed multi-stage ETL pipeline using Spark & Airflow with parallel job processing and scheduled daily runs for automated campaign activation and data movement across storage layers
• Built data lake infrastructure with core tables for customer, loan, and engagement data, aggregating logs from multiple communication channels (SMS, WhatsApp, Email, Calls) for comprehensive engagement tracking
• Developed engagement metrics transformation (MTD/FTD) using Spark Window functions to compute campaign performance, intensity, and customer interaction status metrics across different business levels
• Built PySpark reconciliation script validating data integrity and consistency across multiple storage layers, identifying data discrepancies and missing batches in the pipeline
• Implemented deduplication logic using Window functions and dynamic path resolution for multi-cloud environments (AWS S3 & GCP), enabling flexible data processing across different cloud sources

Dataverse_KYC at Yubi (2024-03)

• The Dataverse KYC Application is a Django-based application developed for KYC (Know Your Customer) purposes
• This application was developed using python django framework. We have created 3 APIs for crawling and getting information about DL, Passport and Voter ID

Data Modelling – Railway booking System at Aravani Foundation (2022-09)

• Design logical data model for the Railway booking system own database. Created SQL Queries based on Data Modelling.
• Data model for online ticket booking application where passenger log in to the ticket booking system.
• The data model has details like trains information, stations, passenger details and login Information.

B.E. in Mechanical – Vel Tech High Tech Dr.Rangarajan Dr. Sakunthala Engineering College (2023)