Prashant Atule
Data Engineer | Databricks | Azure Data Factory | Spark | PySpark | Python | Azure | SQL
Hire this person
Send a job offer directly to this candidate
About
Accomplished Data Engineer with expertise in Databricks. Successfully led data modeling initiatives and optimized ETL processes, enhancing data quality and user experience. Adept at problem-solving, I consistently delivered on-time results, driving project success in the banking and finance domain.
Experience
Software Engineer at NCS Group — National Australia Bank (NAB) (2023-12 – 2026-01)
Led data modeling initiatives and optimized ETL processes for banking services system
- Spearheaded data modeling efforts and developed comprehensive mapping sheets to guide seamless Source-to-Target data transitions.
- Re-engineered ETL pipelines and legacy Hive/Spark jobs for improved efficiency using optimized PySpark and Python solutions.
- Developed and orchestrated ETL workflows using Azure Data Factory ensuring seamless data migration.
- Executed Data Manipulation Language (DML) operations within Azure Databricks workflows.
- Implemented Databricks pipelines from Bronze to Gold layers using connectors and applied complex transformations and aggregations using Jenkins.
- Resolved production issues by identifying and correcting data based on business rules.
- Contributed to developing a robust banking services system tailored for NAB customers.
Software Engineer at NCS Group — IRIN (2022-04 – 2023-12)
Data analysis and quality management for income tax system development
- Identified and corrected data based on project-specific business rules.
- Used MS SQL Server for data analysis and provided on-demand data to development and testing teams.
- Performed data profiling and executed data quality and reconciliation checks.
- Contributed to building an advanced, user-friendly income tax system for Singapore taxpayers.
Associate Engineer at Tech Surya IT Solution (2021-03 – 2022-04)
Data integration and validation for Hadoop ecosystem
- Imported data into Hive from multiple relational databases including Oracle, Sybase, AS400, and SQL Server using Sqoop.
- Engaged with business analysts to understand requirements and translate them into technical designs.
- Implemented data validation, data quality checks, and profiling processes.
- Performed incremental data loads to populate Hive external tables and HDFS.
Education
Bachelor of Engineering in Electrical – Dr. Babasaheb Ambedkar Marathwada University (2014 – 2017)
