Dnyanesh Patharwat
Big Data Engineer
Hire this person
Send a job offer directly to this candidate
About
Data professional with 2 year of experience, skilled in Hadoop, PySpark, SQL, Hive, Python, R programming with proven exposure to data migration projects at NSE. Successfully migrated Green Plume data to the Cloudera Data Platform (CDP) using Hive, Informatica, and related tools. Strong track record in delivering robust data solutions and optimizing ETL processes to achieve scalable, high-impact results.
Experience
Big Data Engineer at National Stock Exchange (NSE) (2024-10 – Present)
- Wrote and optimized complex SQL queries for large-scale data processing
- Developed automation scripts using Unix/Linux shell scripting for data workflows
- Applied data warehousing fundamentals in designing and maintaining data pipelines
- Built and managed ETL/ELT pipelines for efficient data ingestion and transformation
- Worked extensively with relational databases and SQL-based systems
- Utilized SQL and NoSQL concepts for handling structured and semi-structured data
- Performed data modeling, data mapping, and data mining for analytics use cases
- Optimized SQL query performance using indexing, partitioning, and tuning techniques
- Developed data processing logic using Python (PySpark)
- Applied Big Data technologies such as Hadoop and Spark for distributed data processing
- Worked on data processing pipelines for large-scale datasets
- Familiar with cloud platforms such as AWS
Education
Advanced Computing Program – Centre for Development of Advanced Computing (CDAC), Mumbai (2023-01 – 2024-12)
– D.Y. Patil School of Engineering, Ambi, Pune (2020-01 – 2023-12)
– D.Y. Patil Polytechnic, Akurdi, Pune (2017-01 – 2020-12)
