Hemangi Santoki
(Immediate joiner) Data Engineer|Spark|SQL|Python
Talent
Centretown, OttawaMember since June 21, 2023
·
Hire this person
Send a job offer directly to this candidate
About
- Total 6 years of experience in the IT industry and 3 years of experience as a data engineer.
- Proficient in designing and implementing Big Data projects within the Hadoop ecosystem, utilizing technologies such as MapReduce, HDFS, Hive, and Python, and performing data pipeline development and ETL operations.
- Proficient in Spark Architecture, including Data Frame, Dataset, and RDD, as well as the Hadoop HDFS architecture and MapReduce framework.
- Experienced in utilizing AWS services such as S3, EMR, Glue, and Airflow, demonstrating a strong understanding of cloud-based data storage, data processing, and workflow management.
- Strong practical experience with Relational Databases, showcasing hands-on expertise.
- Proficient in databases, Docker, Git, CI/CD, and Agile methodologies, highlighting a well-rounded skill set in essential tools and practices.
Experience
- Led a Healthcare project involving data ingestion into Postgres and Redshift tables, implementing various events based on business requirements to enhance data evolution.
- Actively contributed to the development of 5 number of Spark applications using Python, leveraging advanced data processing capabilities.
- Optimized existing code in Hadoop by utilizing Spark-SQL, Data Frames, and RDD, resulting in a 50% improvement in performance and efficiency.
- Implemented ETL processes by loading CSV data into intermediate tables, performing transformations, and persisting the results in Redshift tables.
- Designed and implemented 6 number of Airflow pipelines to handle diverse data sources and streamline data workflows in the healthcare domain.
Education
Bachelor of Engineering in Computer Science
