Siirry pääsisältöön

Data Engineer

Technology
SGI
Helsinki, Suomi2 kuukautta sitten
Palvelusopimus

Tehtävänkuvaus

Assignment Overview For our customer we are looking for a Data Engineer to migrate SAS workloads to the Databricks Lakehouse Platform. You will translate legacy SAS data processing into PySpark and SparkR, design scalable pipelines, and optimize Delta Lake tables for performance and reliability. Collaborate with analysts and data scientists to ensure functional equivalence between SAS and Databricks implementations, implement CI/CD with Git and Azure DevOps, and validate data through automated testing and data quality checks.

Key Responsibilities

Migrate SAS workloads to the Databricks Lakehouse Platform.

Translate legacy SAS data processing into PySpark and SparkR

Design scalable pipelines

Optimize Delta Lake tables for performance and reliability

Collaborate with analysts and data scientists to ensure functional equivalence between SAS and Databricks implementations

Implement CI/CD with Git and Azure DevOps, and validate data through automated testing and data quality checks.

Required Experience Mandatory

Databricks

Good experience to have

PySpark

SAS, Basics

ETL

SQL

Additional Information

Language requirements:h

Finnish mandotary

English

Start date: As soon as possible after candidate is choosen

By applying for this role, you consent to SGI contacting you by telephone regarding recruitment services, market updates, and relevant business opportunities

We believe in equal opportunity for all and actively encourage applications from diverse backgrounds, experiences, and perspectives.

Source Group International Ltd is acting as an Employment Business in relation to this vacancy

Keywords
OpenGLDevOpsSqlCI / CDGitCI/CD

Kiinnostaako tämä tehtävä?