Data Engineer

We're hiring on behalf of a Haystack partner!

The Role

• Support a strategic modernization initiative to migrate SAS datasets, ETL processes, and analytical workloads to the Databricks Lakehouse Platform
• Convert legacy SAS data processing into scalable PySpark and SparkR solutions
• Develop scalable data pipelines using Apache Spark within the Databricks environment
• Design and optimize Delta Lake tables for performance, reliability, and scalability
• Validate migrated datasets through automated reconciliation and data quality testing
• Partner with business analysts, data scientists, and application teams to ensure functional equivalence between SAS and Databricks solutions

What You'll Need

• Bachelor's degree in Computer Science, Information Technology, Engineering, Mathematics, or a related discipline
• 5+ years of experience in Data Engineering or ETL development
• Strong experience developing with SAS (Base SAS, DATA Step, PROC SQL, SAS Macros)
• Hands-on experience with Databricks and proficiency in PySpark and/or SparkR
• Advanced SQL development skills and experience with Delta Lake architecture and Spark optimization techniques
• Experience building and maintaining enterprise-scale data pipelines, knowledge of data modeling, data warehousing, and distributed computing principles

What's On Offer

• Opportunity to work on enterprise SAS modernization initiatives
• Exposure to Unity Catalog, Delta Live Tables, and Databricks Workflows
• Familiarity with Azure Data Factory, Azure Synapse, or other cloud-native data integration services

Apply via Haystack today!