Rudra Prasad Nahak

Data Engineer with 3+ years of experience in designing, developing, and optimizing scalable data pipelines for healthcare and enterprise data platforms. Proficient in Databricks, PySpark, Apache Spark, SQL, Python, Delta Lake, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS Gen2), Oracle, and Azure DevOps. Experienced in building end-to-end ETL/ELT pipelines, implementing Lakehouse architectures, and processing large-scale structured, semi-structured, and streaming data.

Strong expertise in Delta Lake, Spark SQL optimization, data modeling, workflow orchestration, performance tuning, and data quality validation. Skilled in applying partitioning, caching, broadcast joins, AQE, Z-Ordering, Liquid Clustering, and schema evolution to improve processing efficiency and reduce execution time.

Data Engineer at OPTIWIST TECHNOLOGIES Private Limited (2023-02 – Present)

Project: Migration of Data Ingestion Pipeline from Oracle to Databricks

• Designed and implemented end-to-end ETL pipelines in Databricks (PySpark, Delta Lake, SQL) to process large-scale healthcare datasets including patient records, claims, lab results, and billing data.
• Integrated data from multiple sources — Oracle RDBMS, CSV/JSON flat files, and HL7/FHIR APIs — into Delta Lake, applying schema validation, deduplication, and null handling to ensure data quality and compliance.
• Built data validation frameworks achieving 99%+ accuracy across ingestion workflows.
• Developed automated workflows in Databricks for scalable and reliable data processing.
• Optimized data pipelines to improve processing efficiency and reduce latency for hospital analytics use cases.
• Collaborated with BI and analytics teams to deliver personalized patient-care insights and improve decision-making for hospital management

MCA – Khallikote Unitary University (2023 – 2026)