Pallavi A M
Data engineer
Hire this person
Send a job offer directly to this candidate
About
Big Data Engineer with 3 years of relevant experience, Proficient in Python,
SQL, and Spark, with expertise in designing and implementing data migration projects. Successfully migrated complex HQL scripts into Pyspark on Palantir
Foundary, ensuring seamless data transfer and ETL processes.
Experience
Led end-to-end implementation of Mcloud Migration into Palantir Platform, including design,
requirement gathering, and build, resulting in improved data processing efficiency.
Successfully migrated complex HQL scripts into Pyspark on Palantir Foundary, enabling seamless data transfer from SQL and Hive to PySpark and reducing processing time.
implemented robust code repository, code workbook, and data lineage, ensuring efficient team collaboration and easy maintenance of codebase.
Developed monitoring and health check systems for datasets, reducing data-related errors and enhancing data reliability.
Education
2015 - 2019| Maharaja institute of technology, Mysore
