Numerator - Senior Data Engineer - Python/PySpark

About the Role :

Key Responsibilities :

• Design, build, and maintain scalable data ingestion and transformation pipelines using PySpark and Python.
• Implement and manage data processing workloads on Databricks, ensuring optimal performance and cost efficiency.
• Monitor data pipeline health, troubleshoot issues, and implement proactive reliability enhancements.
• Document data models, pipeline designs, and operational procedures for knowledge sharing and compliance.

Requirements :

• 7 - 10 years of experience in data engineering roles
• Experience in Pyspark and Python
• Experience working with Microsoft Azure Cloud
• Experience in Agile methodologies (Scrum/Kanban)
• Experience with PySpark, Databricks
• Experience working with Devops pipeline, preferably Azure DevOps

Preferred Qualifications :

• Bachelor's or master's degree in computer science, Information Technology, Data Science, or a related field.
• Experience working in a support focused role
• Knowledge/experience in AI ML techniques
• Certification in relevant Data Engineering discipline or related fields.