RiDiK - Data Engineer - Python/PySpark

Responsibilities :

• Design, develop, and maintain scalable data pipelines and backend services using Python.
• Build and optimize ETL/ELT workflows using PySpark, Spark SQL, and Azure Data Factory (ADF).
• Develop and manage data ingestion, transformation, and processing solutions on Databricks.
• Implement and maintain Delta Lake architectures to support reliable and high-performance data platforms.
• Write efficient and optimized SQL queries for large-scale datasets.
• Collaborate with business stakeholders, architects, data analysts, and cross-functional teams to understand requirements and deliver robust solutions.
• Work with Azure cloud services to deploy, monitor, and maintain data applications.
• Implement CI/CD pipelines and deployment automation using Azure DevOps.
• Perform troubleshooting, performance tuning, and optimization of data workloads.
• Ensure adherence to coding standards, security guidelines, and best engineering practices.
• Participate in solution design discussions, code reviews, and technical documentation.

Tech Stack :

• Python, PySpark, Spark SQL, Databricks, Delta Lake, SQL, Azure Data Factory, Azure DevOps.

Qualifications :

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
• 6 years of relevant industry experience in Data Engineering, Data Platform Engineering, or Backend Python Development.